Create Ubuntu VPS

[wildweaselmi]

Once you have your VPS and the DNS is setup (see the Slicehost dns setup article) we can concentrate on setting up the Operating System.

In this Ubuntu setup guide, we already have a basic Ubuntu image installed by the nice folks at Slicehost or the lovely folks at your VPS provider.

Let's get on, update and secure it.

We'll start with the basics of adding a new user, updating the Ubuntu OS, basic security (SSH configuration and iptables) along with an optimised mysql/ruby-on-rails stack with postfix and subversion.

We'll install screen so we don't have to worry if the connection drops. We'll also create several personal rc files for our convenience.

All code that you will be inputting will be shown in fancy boxes and, for the most part, you can simply copy/paste the code as is. However, some need adjusting to use your IP address and your passwords.

Note:

This also works on other VPSs. It is aimed specifically at a Slicehost VPS but I have used it and tested it on other VPS providers. The only difference is that your OS image may not have some of the base programmes, so if I say 'wget this-remote-file', you may need to install wget first.

let's go:

On your LOCAL computer, edit the SSH known_hosts file and remove any entries that point to your VPS IP address. If this is a brand new VPS then you will not need to do this, but an OS reinstall will result in a different signature.

nano ~/.ssh/known_hosts

If you are not using Linux on your LOCAL computer, the location of the known_hosts file will differ. Please refer to your own OS for details of where this file is kept.

As soon as you have your IP address and password for your VPS login via SSH:

ssh root@YOURIPADDRESS

Now we're logged in to the VPS, immediately change your root password

passwd

Add an admin user (I've used the name paul here but any name will do).

adduser -m paul

As paul is our main administration user (we don't normally log in as root!) he needs to be able to sudo (Super User privileges) so we give those permissions via the visudo command:

visudo

At the end of the file add:

paul ALL=(ALL) ALL

Now we'll create two directories. One in our root user folder and one in our new made paul user folder. These 'hidden' directories (did you note the period (.) at the beginning of the folder name?) will hold the public SSH key so we won't have to enter our SSH password every time we log in.

mkdir /root/.ssh && mkdir /home/paul/.ssh

Now we need to copy (using secure copy) the public key from our LOCAL computer to the VPS. On my LOCAL machine, I have the user onion setup, so I would enter:

scp -2 /home/onion/.ssh/id_rsa.pub root@YOURIPADDRESS:/root/.ssh/authorized_keys

Remember to adjust the path to wherever your OS keeps the public key. This is the only time you'll need to enter the SSH password as the file we just copied over will authorise us to SSH in without it. If you are concerned about security (and passwordless logins are far more secure than using passwords), then here is a nice ssh tricks article.

Now we need to configure SSH to make it more secure.

nano /etc/ssh/sshd_config

Use this ssh configuration as an example.

The main things to change (or check) are:

Port 30000 <--- change to a port of your choosing

Protocol 2

PermitRootLogin no

PasswordAuthentication no

X11Forwarding no

UsePAM no

UseDNS no

AllowUsers paul

Now we're going to do a bit of copying and changing permissions of some files. This is all so that we can log in without passwords later on and not worry about who can see what. There are a few lines shown below (each line is a separate command), but each one is fairly self explanatory.

Overall, we're going to copy the authorized_keys file from root into paul's .ssh folder. Then change the permissions so it's not available to everyone. (Having said that, permissions are not that important as it is the public key, but we don't want anyone writing to the file).

cp /root/.ssh/authorized_keys /home/paul/.ssh/authorized_keys

chown -R paul:paul /home/paul/.ssh

chmod go-w /root/

chmod 700 /root/.ssh

chmod 600 /root/.ssh/authorized_keys

chmod go-w /home/paul/

chmod 700 /home/paul/.ssh

chmod 600 /home/paul/.ssh/authorized_keys

Right, now we have the basics of logging in and security done. Next thing is to set up our iptables so we have a more secure installation. We're going to have four ports open: ssh, http, https and a port for our Litespeed Server GUI.

We're going to create two files, /etc/iptables.up.rules and /etc/iptables.test.rules. One will be the 'permanent' set of rules and one will be the temporary (test) set of rules.

Let's save any existing rules to /etc/iptables.up.rules:

iptables-save > /etc/iptables.up.rules

Now let's see what's running at the moment:

iptables -L

You will see something similar to this:

Chain INPUT (policy ACCEPT)

target prot opt source destination

Chain FORWARD (policy ACCEPT)

target prot opt source destination

Chain OUTPUT (policy ACCEPT)

target prot opt source destination

As you can see, we are accepting anything from anyone on any port and allowing anything to happen. One theory is that if there are no services running then it doesn't matter. I disagree. If connections to unused (and popular) ports are blocked or dropped, then the vast majority of script kiddies will move on to another machine where ports are accepting connections. It takes two minutes to set up a firewall - is it really worth not doing?

Let's assume you've decided that you want a firewall. Lets create the file /etc/iptables.test.rules and add some rules to it.

nano /etc/iptables.test.rules

Look at this example iptables configuration file. You must change the configuration to include your full user connection details. This configuration allows connections to ports 80 and 443 (the standard web server ports) from anyone (which is a good thing, we want people to see our websites!) but only allows connections from the specified IP to the SSH port and to the Litespeed GUI.

So not only do we use non-standard ports for ssh, passwordless logins, no root login, we also only allow connections to the two administration ports (ssh and Litespeed admin) from one specific user.

Not bad, eh?

NOTICE!

If you do not have a static IP then do not setup specific users in the iptables file.

The example file above is in fairly clear sections. Have a careful look at the ssh section, it only allows a connection from one trusted user and drops the rest. So even if someone did know the ssh port we were using it won't accept connections from them. If (somehow!) somebody connects to the ssh port, there is an additional 1 minute lock for every incorrect login attempt. That's as well as not allowing password login at all. The user must have our private key.

You will begin to see the pattern of the iptables configuration file. It isn't complicated and it is very flexible and you can change and add ports as you see fit.

What's the litespeed admin port all about? The web server we will setup later on uses a particular port for administration and configuration. We'll setup the port later when we install the server, but for now, we are just defining the firewall rules.

Good. Defined your rules? Then lets apply those rules to our server:

iptables-restore < /etc/iptables.test.rules

Let's see if there is any difference:

iptables -L

Notice the change? (If there is no change in the output, you did something wrong. Try again from the start). Have a look at the rules and see exactly what is being accepted, rejected and dropped.

Once you are happy with your /etc/iptables.test.rules file and are happy with the output of iptables -L, it's time to save our rules permanently:

iptables-save > /etc/iptables.up.rules

Now we need to ensure that the iptables rules are applied when we reboot the server. At the moment, the changes will be lost and it will go back to allowing everything from everywhere.

Open the file /etc/network/interfaces

nano /etc/network/interfaces

Add a single line (shown below) just after 'iface lo inet loopback':

...

auto lo

iface lo inet loopback

pre-up iptables-restore < /etc/iptables.up.rules

# The primary network interface

...

As you can see, this line will restore the iptables rules from the /etc/iptables.up.rules file. Simple but effective.

Now we have our firewall humming along and we've set the ssh configuration. Now we need to test it. Reload ssh so it uses the new ports and configurations:

/etc/init.d/ssh reload

Don't logout yet...

On your LOCAL computer, open a new terminal and try and login using the administration user (in this case, paul) to the new port number you configured in the sshd_config file and allowed to be open in the iptables configuration:

ssh -p 30000 paul@YOURIPADDRESS

The reason we use a new terminal is that if you can't login you will still have the working connection to try and fix any errors. Slicehost also has the excellent ajax console so if it all goes horribly wrong, you can log into your VPS from the Slicehost management area.

If all goes well you should login without a password to a plain terminal:

paul@yourvpsname:~$

Success!

We now know that the firewall works, the ssh_config works and we can login from the defined location.

First thing is to confirm what OS we're using. We know we should be using Ubuntu LTS but let's check:

cat /etc/issue

You should get an output similar to this:

#Ubuntu 6.06 LTS \n \l

Now let's see how our memory usage is doing (the -m suffix displays the result in MB's which I find easier to read).

free -m

It's nice to know what is going on so let's look at that output:

. total used free shared buffers cached

Mem: 254 68 185 0 2 38

-/+ buffers/cache: 27 227

Swap: 511 0 511

The important line is the second one as the first line includes cached memory - in this test VPS I have 254MB memory in total with 27MB actually used, 227MB free and no swap used. A good start I think.

Let's make the terminal a bit prettier and bit more useful by adding a few lines to our .bash_profile file.

nano ~/.bash_profile

We need to add a few lines at the end of the existing text. The following line will make the terminal show the server name in colour and display the working directory (the directory we are in) in a different colour:

export PS1='\[\033[0;35m\]\h\[\033[0;33m\] \w\[\033[00m\]: '

Now we'll add some aliases to the file. Aliases are simply short cuts to commands or sequences of commands. I've included a few below but you can have as many or as few as you want.

alias free="free -m"

alias update="sudo apt-get update"

alias install="sudo apt-get install"

alias upgrade="sudo apt-get upgrade"

alias remove="sudo apt-get remove"

alias gem="sudo gem"

The examples above are pretty simple. Instead of typing 'free -m' every time I want to look at the memory usage, I just type 'free. Typing 'sudo apt-get install' can get tedious, so I just type 'install'. I still need to provide my password for the command to work, but it is more productive/quicker/easier to have short cuts.

To make our terminal use the changes we can either log out and log in again (a bit much!) or issue the following command:

source ~/.bash_profile

You should now see the VPS name in purple and the working directory in brown. To change the colours to your choosing, adjust the 0;35m and the 0;33m values in the 'export PS1' line of your .bash_profile. For example:

export PS1='\[\033[0;32m\]\h\[\033[0;36m\] \w\[\033[00m\]: '

would give you a green and blue output.

Fun time over! We need to update the software catalogue now, but first some adjustments need to made the sources list that the OS uses. So open up the file:

sudo nano /etc/apt/sources.list

I've put up an example sources list. I simply delete the contents of the original and replace with the list shown in the link. You can add or delete sources as you see fit. A word of warning about adding a 'backports' repository: they are unsupported and we're building a server here, not a desktop - we need stability and durability.

Once the sources.list is updated, issue the command:

sudo apt-get update

NOTE: If you have used the .bash_profile shown above, you do not need to give the whole command. You just need to enter 'update' and the alias will give the entire command. I've put the whole thing here so you know what is happening. Use either the alias you have set up or the whole command.

From the terminal output you will see the download of the updated package list.

The first thing we need to do is install some language packs. In my case I install the english ones. You may have to change it to your chosen language. We do this before updating anything else so the install does not throw up warnings and errors about locations and so on.

sudo apt-get install localeconf language-pack-en

NOTE: I won't mention it again (honest!), but if you use the .bash_profile aliases, you only need to type in 'install package_name' and not the whole command shown above.

Review what it's going to install and press 'Y'.

The install will ask a couple of questions: Answer 'yes' to both.

Now let's update the installed software to the latest Ubuntu LTS ones. The LTS is important here - Ubuntu LTS has guaranteed 5 years of security updates to all the programmes. Remember that we are building a server here and not a desktop and we need that long term stability.

sudo apt-get upgrade

Have a look at what is going to happen and press 'Y'.

That might take a little while to finish and when it is done we're going to reboot the server. It's the only time this will happen and we need to do it as there may have been some significant changes in the base/minimal install. It simply ensures that all the init files and configurations are running nice and smoothly.

sudo shutdown -r now

That command tells it to shutdown, reboot and to do that right now!

You will be 'thrown' off the terminal. Wait a minutes or two (it doesn't take long) and log back in again. If you want, you can also see the output of the reboot via the Ajax console in the Slicehost control panel. Just login and click 'console' next to your slice name. It's not essential to view it, but if you are new to servers, it can give a good idea of what happens in the boot process and may give an idea if something goes wrong. Obviously, you can view the logs as well, but this is a nice and easy way of seeing the 'live action'.

My reboot took about 15 seconds. Now log back in:

ssh -p 30000 paul@YOURIPADDRESS

First thing to do is install 'screen'. This is a great utility that allows us many 'virtual' terminals in our single terminal window. It also has the ability to start a process and be left to it. The connection could drop or you can log out and log in again later to see how the process is doing. A 'normal' ssh connection will not do that - if the connection drops, the process stops which is very annoying when you're an hour into compiling GCC, as you have to start again!

So, let's install screen:

sudo apt-get install screen -y

Did you notice the -y at the end of the command? This tells the install to go ahead and not ask if I want to carry on. Not always recommended but it's safe here.

If you've never used screen before, have a look at this screen tutorial which also includes how to create a .screenrc file so it start exactly how you want it to (my screen session starts with 3 terminals, one running 'top, one main work terminal and a spare terminal for other use).

Let's start simply:

screen

Press the 'Space Bar' and it looks just like the normal terminal, except the colour has gone... Simply issue the 'source' command:

source ~/.bash_profile

Now we are back to using our aliases and useful bash prompt.

If you are not logged in already then log in using the username we setup earlier:

ssh -p 30000 paul@YOURIPADDRESS

Start a screen session:

screen

Ubuntu has some handy meta-packages that include common programmes needed for one purpose. So instead of installing a dozen different package names, you can install just one meta-package. One such package is called 'build-essential'. Issue the command:

sudo apt-get install build-essential

Notice the programmes that are to be installed include gcc, make, patch, headers and so on. All these are needed for many other programmes to install properly. A neat system indeed. Click 'Y' and install them.

mysql install

Right, let's get on and install mysql and then optimise it for use with a VPS. It needs optimising as the default configuration uses a fair bit of memory and there is no need to 'waste' memory like that. We added the mysql bindings for ruby to save time later when we install Ruby on Rails. I like to plan ahead and install packages in groups rather than splinter bit and pieces everywhere and lose track of what is going on:

sudo apt-get install mysql-server mysql-client libmysqlclient15-dev libmysql-ruby -y

mysql installs without a password for the root user (don't confuse this 'root' user with your server's root user - it is simply the mysql user that has access to all the databases). Let's add a password straight away:

mysqladmin -u root password YOURMYSQLPASSWORD

There are some great articles on optimising mysql on the web. A couple to get you started (if you want to know more) are mysql performance and optimising mysql.

Now let's optimise mysql for your VPS. Open the mysql configuration file:

sudo nano /etc/mysql/my.cnf

We're going to add two lines in the basic setting section. Find the first two lines shown below and add the two lines shown after:

language = /usr/share/mysql/english

skip-external-locking

skip-locking

skip-innodb

NOTE: If you intend to use innodb databases then don't include the skip-innodb line!

Next, find the Fine Tuning section and adjust the settings / add the entries as shown below:

# * Fine Tuning

#

key_buffer = 16K

max_allowed_packet = 1M

thread_stack = 64K

thread_cache_size = 4

sort_buffer=64K

net_buffer_length=2K

Before we restart mysql, let's have a quick look at how much memory it's using on the original configuration. Enter the command 'top':

top

Then holding the 'Shift' button down, press 'm'. This will arrange the output of top in order of memory usage. You will see something like this:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

3223 mysql 15 0 ---- ---- ---- -- 0 8.9 0:00.15 mysqld

On this 256MB test slice, mysql is using 8.9% of the memory (I've removed some of the other figures so the output is easily noted).

Exit top (Ctrl and 'c') and restart mysql:

sudo /etc/init.d/mysql restart

Now look at top again. My output is now:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

3453 mysql 15 0 ---- ---- ---- -- 0 2.0 0:00.03 mysqld

See the immediate difference? Memory use is now just 2.0%. When memory is a factor, such as when using a VPS, optimising any install can be a very good idea.

Ruby on Rails install

We'll begin by installing ruby from the apt repositories and then installing rubygems via source code:

sudo apt-get install ruby1.8-dev ruby1.8 ri1.8 rdoc1.8 irb1.8 libreadline-ruby1.8 libruby1.8 libopenssl-ruby -y

We need to create some symlinks from the install to locations every programme would look. Each line below is a separate command:

sudo ln -s /usr/bin/ruby1.8 /usr/local/bin/ruby

sudo ln -s /usr/bin/ri1.8 /usr/local/bin/ri

sudo ln -s /usr/bin/rdoc1.8 /usr/local/bin/rdoc

sudo ln -s /usr/bin/irb1.8 /usr/local/bin/irb

In our home directory, we will create a 'sources' folder:

cd ~

mkdir sources

cd sources

Now we have moved to the sources directory, download the rubygems source code, unpack it and install it:

wget http://rubyforge.org/frs/download.php/20989/rubygems-0.9.4.tgz

tar -xvf rubygems-0.9.4.tgz

cd rubygems-0.9.4

sudo ruby setup.rb

Once done, move back to your home directory and update the ruby components:

cd ~

sudo gem update

sudo gem update --system

Now we can install rails with the command:

sudo gem install rails --include-dependencies

The basic Ruby on Rails install is now completed! I continue the install with imagemagick and a quick ruby test as follows:

sudo apt-get install imagemagick librmagick-ruby1.8 librmagick-ruby-doc libfreetype6-dev xml-core -y

Now we can do a quick test to see if the install works (thanks to brainspl.at for this idea).

The light grey parts of the code shown below are the terminal output - you enter the black/coloured commands:

irb

#irb(main):001:0>

require 'RMagick'

#=> true

#irb(main):002:0>

require 'mysql'

#=> true

#irb(main):003:0>

exit

Obviously, if the output does not return 'true' for each 'require' command, then something has gone wrong. If the result is correct and returns 'true' then we have a working ruby install.

Finally, we need to install postfix (if we don't do this now, other programmes will install exim and all sorts of unneeded email help) and subversion so we can 'check-out' plugins and goodies for our Rails applications:

sudo apt-get install postfix subversion -y

Answer the postfix questions as you see fit but the defaults usually suffice for our purposes as we are only using it to send emails and not for mail-server purposes.

That's it!

We now have a secure, up to date Ubuntu server with mysql, Ruby on Rails and subversion.

At this point, you can split off from these tutorials and install your own server or other needs. However, in the Litespeed tutorials we'll install the Litespeed Webserver - quicker, easier and less hassle than apache or nginx, etc. No messing about with mongrels or trac server daemons. It incorportes all we need in one easy package with an incredibly low memory footprint.

We'll also install the latest php version with our Litespeed server.

[wildweaselmi]

Some comparisons here...

I installed CentOS 5.5 yesterday and it took 5 of the 7 possible CD's and about an hour on an older box

Today I installed Ubuntu 10.10 Server (LAMP and OpenSSH) and it took about 20minutes and used 1 CD. SSH worked the first time, I typed the ip address in my browser and it came up right away.

Downfall... both cpanel and InterWorx both do not support Ubuntu. They favor RedHat, CentOS and cPanel sorta supports Debian (which is what Ubuntu is based off of and so is Mac OS X). So as much as I love cPanel and InterWorx looks promising, they just don't support the right operating system.

From the articles I have been reading, many many many people are tired of CentOS and the lack of packages and all the customizing packages to just get it to work on the O.S. They have all recommended if they could do it again, they would pick Ubuntu for the ease of use and because of all the packages it supports out of the box.

I also tried Webmin, Usermin and Virtualmin and I don't like them. They are difficult and dirty. I suppose for free they are awesome but they definitly aren't any cPanel. It looks like the only option for Ubuntu Server VPS system is Plesk which does support Ubuntu. Now I need to find out if they support intranet and if I can just purchase a license. I don't want to have to pay monthly. Maybe something like InterWorx where you buy the perpetual license for $250 and then you pay $50/6 months if you want updates. That's a deal if you want to keep up with the updates.