I’ve written a simple backup script in Ruby that utilises Rsync I’ve made it public so feel free to fork, change and contribute!

The backups are structured as such:

/remote/location/current - The current folder contains the the latest clone of the intended backup folder.
/remote/location/backup_dir - The backup directory, which by default is the current day (monday, tuesday, etc) contains
the difference to current since it was last run. So it will contain any new files and any files that were deleted.

I need to add a few examples of the settings and how to use it via cron on Linux and Launchd on OS X but the basic script should be working fine.

The script can be found on my github account, http://github.com/RichGuk/rrsync/tree/master

You can simply clone it with the following command:

git clone git://github.com/RichGuk/rrsync.git

Today I decided to setup Xen (not the Half-Life world) on a clean Debian installation so I could have a few VPSes to play around with load balancing mySQL, FastCGI and Mongrel. I thought I’d break down the steps I went though to get the basic Xen setup working.

Note: This is by no means a perfect setup, it’s more of a quick way to play around with Xen using a spare PC in the home. If you want more of a production level setup I suggest checking out the Xen website, or Google.

Step 1. Install Debian (Etch)

Here you just need to setup a standard Debian system, I’m using the 64bit edition of Etch but the process is the same for most Debian releases. I won’t go into detail on how to setup Debian as there are plenty of how-tos out there, for me the normal setup was fine. During the installation I gave the machine judgement as the hostname and gnet.foo as the domain (you can use whatever) I also unchecked desktop from the selectable software menu as I didn’t need/or want a GUI.

Step 2. Configure Debian ready for Xen

Once you’ve booted and logged into your system you need to install a few applications, but before we do that I tend to remove the CD source from apt-get’s sources.list:

vim /etc/apt/sources.list
#comment out the CD line
#deb cdrom:[Debian...........
apt-get update

Now install the packages (remove vim-full if you're not using vim):

apt-get install ssh build-essential vim-full

Next we need to configure the server to use a static IP (if you didn't during the installation). I gave my box (judgement) a static IP of 192.168.1.10. Open up the interfaces file:

vim /etc/network/interfaces

Now if you selected DHCP during the installation your file should look similar to this by default (eth2 was my interface, yours might be different):

allow-hotplug eth2
iface eth2 inet dhcp

Change yours to match this (of course change eth2 to match yours and whatever IP you wish to use):

auto eth2
address 192.168.1.10
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

Once you've done that you need to restart the networking:

/etc/init.d/networking restart

If all is working then you should be able to ping other computers on your network and they should be able to ping your machine.

If you wish you can continue the rest of the setup via SSH (on windows you can use Putty).

Finally just make sure your system is up-to-date before giving it a reboot:

apt-get upgrade
shutdown -r now

Step 3. Installing Xen

There are two main options when it comes to installing Xen. You can compile it from the source code or simply use apt-get, the latter worked fine for me and is dead simple...to install via apt-get use the following command (bridge-utils is used to setup the network for the virtual machines):

apt-get install xen-linux-system-2.6.18-6-xen-amd64 bridge-utils

The version of Xen kernel might of changed since writing this, try this command to see what's there:

apt-cache search xen-linux

When that has finished installing give the machine a reboot and select Xen kernel from the GRUB menu (for me it became the default option):

shudown -r now

Once the system is back up and logged in (your previous credentials should work) check that you've booted the correct kernel:

uname -r

It should read something like 2.6.18-6-xen-amd64 if not then you've most likely booted the wrong kernel from GRUB.

Step 4. Creating virtual images or DomU's (the VPSes)

To help with creating, starting, stopping, etc... you need to install xen-tools:

apt-get install xen-tools

Next we need to create the place we want to store the domains (virtual servers), I put mine in /home/xen (xen requires a folder called domains within this folder) so run this command:

mkdir -p /home/xen/domains

Once you've created the location to store the domains you need to edit a few options in Xen's configuration. Start by opening up xend-config.sxp:

vim /etc/xen/xend-config.sxp

and look this for line (around line 70 for me):

##
# To bridge network traffic, like this:
#
# dom0: fake eth0 -> vif0.0 -+
#                            |
#                          bridge -> real eth0 -> the network
#                            |
# domU: fake eth0 -> vifN.0 -+
#
# use
#
# (network-script network-bridge)

and uncomment the network-script network-bridge part so it looks like this:

##
# To bridge network traffic, like this:
#
# dom0: fake eth0 -> vif0.0 -+
#                            |
#                          bridge -> real eth0 -> the network
#                            |
# domU: fake eth0 ->; vifN.0 -+
#
# use
#
(network-script network-bridge)

Now we need to setup some defaults as well as the location to store the domains by opening up xen-tools.conf:

vim /etc/xen-tools/xen-tools.conf

There are several things to comment out and change here so I'll just list the values I changed in mine:

...
dir = /home/xen
...
debootstrap = 1
...
dist = etch
#If you want to give your domains static IP's rather than use DHCP (which I did then you can setup the default gateway/netmask)
gateway = 192.168.1.1
netmask = 255.255.255.0
...
#This bit is important, as mine differed to the default you need to change kernel and initrd to point to the right ones (ls /boot/* - to find out)
kernel = /boot/vmlinuz-2.6.18-6-xen-amd64
initrd = /boot/initrd.img-2.6.18-6-xen-amd64
...
#Change this to your prefered mirror (for me that's the UK):
mirror = http://ftp.uk.debian.org/debian/

Restart Xen to make sure the configuration loads correctly:

/etc/init.d/xend restart

You should now be ready to create your first domain. If you're happy with the defaults in the xen-tools.conf file then you can leave a lot of the options out or you can of course put them as arguments of the command, check the man pages for xen-create-image for more information:

xen-create-image --hostname=i-am-webial --ip=192.168.1.11 --netmask=255.255.255.0 --gateway=192.168.1.1 --passwd

You should see output of something like this:

General Infomation
--------------------
Hostname       :  i-am-webial
Distribution   :  etch
Fileystem Type :  ext3

Size Information
----------------
Image size     :  4Gb
Swap size      :  128Mb
Image type     :  sparse
Memory size    :  128Mb
Kernel path    :  /boot/vmlinuz-2.6.18-6-xen-amd64
Initrd path    :  /boot/initrd.img-2.6.18-6-xen-amd64

Networking Information
----------------------
IP Address 1   : 192.168.1.11
Netmask        : 255.255.255.0
Gateway        : 192.168.1.1

Creating swap image: /home/xen/domains/i-am-webial/swap.img
Done

Creating disk image: /home/xen/domains/i-am-webial/disk.img
Done

Creating ext3 filesystem on /home/xen/domains/i-am-webial/disk.img
Done

Installing your system with debootstrap mirror http://ftp.uk.debian.org/debian/
....
....
....

Once it's finished you can start your domain by using the following command:

xm create /etc/xen/i-am-webial.cfg

Extra Stuff

Open a console for the domain:

xm console i-am-webial

you can exit the console by doing Ctrl + ].

To shutdown a domain:

xm shutdown i-am-webial

Increase the RAM of the domain:

vim /etc/xen/i-am-webial.cfg

find and edit memory

memory  = '512'

Resize the swap to 1GB (Resize Xen DomU swap):

cd /home/xen/domains/i-am-webial
dd if=/dev/zero of=swap.img bs=1024k count=1024
mkswap swap.img

Expand the file system by 4 GB (Resize Xen DomU disk)

dd if=/dev/zero bs=1GB count=4 >> disk.img
resize2fs -f disk.img

Hopefully you should now all be setup with basic VPSes, if you need any help with problems please feel free to contact me. You may also leave any feedback/comments you wish below.

So the other day I decided to switch from using Apache to Nginx not because apache isn’t any good, far from it. Nginx just uses much less ram and when you’re on a fairly small slice, using up more ram just isn’t good. I was going to benchmark speed and things between apache and nginx but I forgot to take some readings before stopping the apache service and installing nginx so we’ll skip over that idea like the thought never came into my head.

I’ll mention a few things I like about nginx first:

1. It’s Russian -Being Russian it makes me feel like a communist, which is fun. It also reminds me of being build to take all things thrown at it.
2. It uses a lot less memory - As I said before, not really done much benchmarks so these were some quick figures from ‘top’ but I had apache running with about 5 instances each using 2-3% of ram even at 2% each that’s still 10% of ram being used just for apache… with nginx I appear to have 2 instances running both using 0.2% of ram, so 0.4% in total - which is a nice difference.
3. Faster page serving - Now this is meant to be a server fast at serving static content so its not just be me but I find the site much quicker compared to when it used to be on apache (all the sites in fact). I’m not to sure if I just think it’s quicker, the slice has more ram free so never going into swap or…. it is just a lot quicker at serving pages.

Read the rest of this entry »

Off to see Kerry at aber this weekend so thought I’d pump out a quick youtube Friday before I head off.

First off I thought I’d start with the Floyd, this is from division bell, which is one of my favourite Pink Floyd album.

Next up a bit of Thom, not to sure if I’ve posted this before - I have yet to import posts from the old blog. I’m sure you can survive having to watch it again…..after all it is Thom

Finally I thought I’d post this video I found, I was orginally going to find Bob’s version, and then the one thats playing at the end of the last season of Battlestar Galactica but instead I found this insane child playing it like a pro. Enjoy (All along the watch tower btw).

Lately I’ve become fanatical over web frameworks, mainly Django, Rails and CakePHP. Something about them just clicks, maybe its because I can go back to a rails app months after I’ve written it, even with no comments, and just know where each file is likely to be and where to find a certain method. Also I can create CRUD (Create, Read, Update and Delete) functionality in seconds as most frameworks provide some form of scaffold, something that would take time to produce without a framework in something like PHP.

If there is one thing that pisses me off about frameworks - it’s the people who don’t like them. If like me you’ve read a lot of people talking about how crap frameworks are? How they don’t see the point: “What’s the point in a framework? I can just find all the different library’s I need such as templating, databases and include them in my app”. Why? Why would you want to waste your time including a database class, setting it up and then going to find a template engine to include and then setting that up too! When the framework will just give you that from the word go. It just confuses me. Then there is the fact that with most frameworks you get nice clean URL’s, free without having to do anything. try doing that in your simple PHP app without doing some form of mod_rewrite YOURSELF.

Most frameworks will log errors, show debug information (in development mode), give you the option of simply editing a file (not having to set it up too) to get nice 404/500 type errors. Something that would require you to make your own form of exceptions, your own error logging if you were not using a framework, which would do it all for you.

Another thing that pisses me off about not having a framework - is validating form inputs. Argh, it annoys me so much having to do lots of if statements to check if a field is empty, having to do a query to check a field isn’t already taken. In a framework, for example rails I just have to do:

validates_presence_of :title

Tell me again why people prefer to spend ages writing if statements and queries to check form data?

People will always try to say “I don’t like being told how I should code” - Fuck me, yes maybe…maybe there are some standards you have to stick to in frameworks and if you’re totally mentally retarded you might think this is horrible idea to have to agree on a naming convention for the benefit of others. It’s like these people have never worked with other people? At work we’ve had entire meetings just agreeing on a standard form of code to use, have you ever tried submitting a PEAR class?! talk about anal. I’ve found apart with some naming conventions you can pretty much code however the hell you like in a framework - much like you can if you were not using a framework!

From my experience so far I’ve never had a website that has not fitted the framework type model… now I’m sure there are loads of sites that would not work at all in a framework and the anti-framework people will be the first to tell you but that’s fair enough if it doesn’t fit the model then you don’t have much choice. But these people do it out of choice?! Are they better than us mortals? Do they know something we don’t? No! from my experience the people who shoot down using a framework to speed up development, to help bring out creativity, to help with debuging, these people are generally scared to learn something new, scared to well…learn. They refuse to even look becuase they know it’ll be something else that’s not what they know, even if the current method they use is rubbish.

All this technology, all the advances in medicine we have, space travel, you name it. It all comes from people wanting to discover, people wanting to learn. So I beg you, don’t just push the framework out the window as though it’s all ‘hype’ and doesn’t have any real world use.. please please please just give it a try. There’s a reason it’s popular, even among the big names, because frameworks are damn good.