Force Apache2 to redirect from HTTP to HTTPS

Want to redirect requests for http://www.yoursite.com to https://www.yoursite.com?

There are many suggestions out there to add a .htaccess file with rewrite conditions[1] but I figured I’d suggest a second, through a simple change to your httpd.conf file (or if you are running Ubuntu or another distro that splits the httpd.conf file into multiple files, in your /etc/apache2/sites-available/{yoursite} configuration file. (If you are running a pretty Ubuntu install, the file is /etc/apache2/sites-available/default)

This technique still uses the rewrite engine (so you’ll need mod_rewrite module) but it places the configuration in the httpd.conf file (or its equivalent) and out of the .htaccess file.  There are many reasons you might want to do this, such as prevent it from being changed (many site configurations allow users to edit all .htaccess files but prevent them from editing the httpd.conf file) or to prevent it from being overwritten by certain web application packages (many application packages including WordPress and MediaWiki employ custom .htaccess files to provide more friendly URLs).

The change is simple, in your httpd.conf file, change the following part of your virtual host section:

<VirtualHost *:80>
        ServerAdmin contact@yourwebsite.com

        DocumentRoot /var/www/public

        <Directory />

… to something resembling the following:

<VirtualHost *:80>
        RewriteEngine on
        ReWriteCond %{SERVER_PORT} !^443$
        RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [NC,R,L]
</VirtualHost>

<VirtualHost *:443>
        ServerAdmin contact@yourwebsite.com

        DocumentRoot /var/www/public

        SSLEngine On
        SSLCertificateFile    /etc/apache2/ssl/mycert.pem
        SSLCertificateKeyFile /etc/apache2/ssl/mycert.key

        <Directory />

The important things here are that the port 80 Virtual Host has been changed to listen on port 443 (Line 7), there is a new Virtual Host that has been added above our existing Virtual Host that is set to listen on port 80 and only contains the rewrite code (Lines 1-5), and finally, there are lines added that enable and configure the SSLEngine (settings may differ based on implementation)(Lines 12-14).

Restart Apache and your browser should redirect requests to http://www.yoursite.com/ to https://www.yoursite.com.  In case you’re wondering, a request for http://www.yoursite.com/staff/about.php will also redirect to https://www.yoursite.com/staff/about.php.

Leave the first comment

Clearing Subversion information from a project

Subversion stores its information in hidden folders named “.svn” inside each added directory. Sometimes, you might want to remove all off these .svn directories, and start fresh. There are any number of reasons to want to do this, but in my case I wanted a quick way to reset a working copy on a windows machine.

I found a good batch script (source) that does exactly what I needed:

FOR /F "tokens=*" %%G IN ('DIR /B /AD /S *.svn*') DO RMDIR /S /Q "%%G"

I just threw that line in a text file, and saved it as rmsvn.bat in the highest directory that I wanted to strip .svn files from.

Opening up a command prompt, changing to the directory I saved the batch file in (with cd) and then running the file (with rmsvn.bat) worked perfectly.

Leave the first comment

Moving a Subversion Repository

Sometimes it is necessary to move a subversion repository. In this case, it was because I wanted to consolidate a few subversion repositories into one directory on a dedicated hard drive.

Moving a subversion repository usually consists of two steps:

  1. Moving the repository
  2. Updating the working copy to reflect the repository move

This post will handle both steps.

Move the repository

Instead of actually moving the repository as one would expect, we’re going to backup our existing repository, import out backup into a newly created one.  First we’ll list the commands, then break down what we’re doing.

svnadmin dump /path/to/old-repository > ~/repo.dmp
svnadmin create /path/to/new-repository
(Update file/directory permissions)
(Update repository permissions)
svnadmin load repository-name < ~/repo.dmp
  1. L1: We dump the repository into a backup file.
  2. L2: Create a new repository.
  3. L3: Update file / directory permissions (by using some combination of chown, chgrp and chmod)
  4. L4: Update repository permissions (by editing files like /path/to/new-repository/conf/passwd and /path/to/new-repository/conf/svnserve.conf)
  5. L5: Extract the backup into the new repository.

Update the working copy

Once we’ve moved our repository, we now need to update the working copy so that commits and updates will reference our new repository.

cd /path/to/working-copy
svn info
svn switch --relocate svn://hostname/path/to/old-repository svn://hostname/path/to/new-repository
svn info
  1. L1: We change to the path of the working copy.
  2. L2: Display the path to the old repository through the svn info command.
  3. L3: Switch the working copy’s repository.
  4. L4: Display the path the the working copy’s repository (verify that the switch worked) through the svn info command.

Clean up

Now that we’re done moving the repository, we want to verify the move and clean up.  To verify the move, we can just check the output of the svn info command.  To further verify, we can run svn update to ensure that nothing is updated.

We will probably also want to remove both the old repository and the backup file.  A simple rm -r for the repository and rm for the backup file is fine.

Lastly, its possible that you would want to use a trimmed down set one of these steps. For example, if you didn’t care about keeping the version history and just wanted to move the repository location, you could skip steps 1 and 5 in the first section. Also, if you didn’t care about moving the working directory but just wanted to make a complete copy of the repository including the version history, you could complete the whole second section.

2 comments so far, add yours

ping fails but dig / nslookup works?

I’ve now run into two scenarios where this happened. The most recent was after I had to kill a vpnc process (used to connect to a cisco VPN).

It looks like this:

[james@workstation ~]$  ping google.com
ping: unknown host google.com

[james@workstation ~]$  dig google.com +short
72.14.204.147
72.14.204.99
72.14.204.103
72.14.204.104

What ended up happening was that the VPN was configured to tunnel all traffic through it, so when it re-wrote the /etc/resolv.conf file, it didn’t append the VPN nameservers to the nameservers provided to us by our DHCP lease, but completely overwrote them. I’m assuming that when you closed the VPN it would replace the resolv.conf file with the one containing the non-VPN nameservers but since I killed it, it was not restored.

Anyway, the fix was easy but finding it out was annoying.

All you have to do is release and renew your DHCP lease.

You could try:

dhclient -r; dhclient

… but I was on an SSH connection and I didn’t quite trust the second command to be run after I lost the connection when the lease was released (this seems like a silly fear but whatever).

Instead I wrote a bash script and put it in a file:

#!/bin/bash
#refreshlease.sh
IFACE="eth0"

dhclient -r ${IFACE}
dhclient ${IFACE}

… and ran that over the SSH connection. The connection seemed to get dropped for a few seconds but then came back up. Checking /etc/resolv.conf showed that my original nameservers were, in fact, back and I was able to resolve DNS queries:

[james@workstation ~]$  ping google.com
PING google.com (64.233.169.105) 56(84) bytes of data.
64 bytes from yo-in-f105.1e100.net (64.233.169.105): icmp_seq=1 ttl=238 time=31.1 ms
Leave the first comment

Quick Bash Trick: Looping through output lines

Lets say you want to add line numbers to a text file. For example, you want to see this file:

The quick brown fox
jumps over the
lazy dog

… displayed on your screen list this:

1  The quick brown fox
2  jumps over the
3  lazy dog

Of course, you could use cat -n if you’re using cat and want line numbers, but if you want to, say, add a 4 space indent to the output of iwconfig, the following technique will work.

First, we run the program / script and capture the output:

output=$(iwconfig 2>/dev/null)

Here we’re running iwconfig in a subshell and capturing the output. In this case, I’m redirecting stderr to /dev/null, otherwise iwconfig returns lines like lo no wireless extensions. up through the subshell to the shell thats calling the bash script. I don’t want this so I’m throwing it out.

Next, we loop through the output using a for loop:

for LINE in ${output} ; do

    echo ${LINE}

done

Running this will not get us the output we’re expecting, it will return each space-separated word/phrase on it’s own line. If we are still talking about the pangram above, we would see:

The
quick
brown
fox
jumps
over
the
lazy
dog

The problem is, the IFS (Internal Field Separator) contains the space character.

(Quick Aside)
To see what the IFS is, type:

echo -n "$IFS" | hexdump

My IFS contains 0×09, 0×20, 0x0a (tab, space and newline respectively).
(End Aside)

We can change the $IFS in our script (to only contain the newline character) which lets us iterate through the output where separated by newlines.

First, we save the original IFS and define the newline IFS:

OIFS="${IFS}"
NIFS=$'\n'

Next we set the IFS to our newline IFS:

IFS="${NIFS}"

Then, we loop through the output (in the loop we reset the IFS incase we expect it to include the space, etc. and then set it to newline IFS before our next iteration)

for LINE in ${output} ; do
    IFS="${OIFS}"

    echo "---- ${LINE}"

    IFS="${NIFS}"
done

Lastly, we reset the IFS:

IFS="${OIFS}"

The final code looks like this:

        output=$(iwconfig 2>/dev/null)

        OIFS="${IFS}"
        NIFS=$'\n'

        IFS="${NIFS}"

        for LINE in ${output} ; do
            IFS="${OIFS}"

            echo "---- ${LINE}"

            IFS="${NIFS}"
        done
        IFS="${OIFS}"

… and we get what we had hoped:

---- The quick brown fox
---- jumps over the
---- lazy dog

In closing, it should be noted that I’ve done this a few different ways (including splitting the output with awk / sed) but I like the simplicity of this method. I may try to find the other methods and write them up to if I get some time.

P.S. If you ever need to reset your IFS but don’t want to close the terminal, type this:

IFS=$'\n\t '
2 comments so far, add yours

iGoogle Minimalized

I hated how much space the Google header took up at the top of the page. I searched for themes but none seemed to address the issue. I then looked into developing my own theme through their theme API and it turns out the reason none of the themes address the large header is because Google doesn’t give the developer the option to control that.

All you can really control is colors, add a few custom images and select between icon sets, but nothing having to do with sizing or positioning.

My next step was to examine the DOM and whip up a GreaseMonkey Script that shrunk it. Having some experience with it, I made use of a few jQuery features in the script.

Before:

Before

Before

After:

After

After

Enjoy!

Direct Download

// ==UserScript==
// @name           Minimalize iGoogle Theme
// @namespace      http://www.edwards-research.com
// @description    Minimalize iGoogle Theme
// @author         by James Edwards
// @include        http://www.google.com/
// @include        http://www.google.com/ig
// @require        http://code.jquery.com/jquery-latest.min.js
// ==/UserScript==

// Written by James Edwards
//		Feel free to use this script in any way you like, commercial or otherwise.
//		Attribution is optional but encouraged.
//
// jQuery Integration from: Greasespot
//		http://wiki.greasespot.net/Code_snippets#Use_jQuery_in_a_GreaseMonkey_script
(function() {
	//<div id="nhdrwrap">
		$("div#nhdrwrap").css("padding","0px");
		$("div#nhdrwrap").css("margin","0px");
		$("div#nhdrwrap").css("height","100px");
	//<div id="nhdrwrapinner">
		$("div#nhdrwrapinner").css("background-image","none");
		$("div#nhdrwrapinner").css("height","100px");
		$("div#nhdrwrapinner").css("margin-top","-20px");
	//<div id="regular_logo">
		$("div#regular_logo").hide();
	//<p class="gseaopt">
		$("p.gseaopt").hide();
	//<div class="personalize_link">
		$("div.personalize_link").hide();
	//<div id="header_promo_wrapper">
		$("div.header_promo_wrapper").hide();
	//<div id="promo" class="sandbox_msg" style="margin-top:-20px;"
		$("div.sandbox_msg").hide();
	//<div class="gradient">
		$("div.gradient").hide();

}());
Leave the first comment

Color Schemes for Eclipse CDT – Part 4

I’m proud to announce the alpha release of the Eclipse Theme Generator (available here). This site allows you to design a color-scheme in real-time and see how it looks. After you are happy with your theme, you can proceed to export the settings into the appropriate files.

Eclipse CDT Theme Generator

Some things to note:

  • This is just an alpha release, seriously, there are many shortcomings and I’m sure a few bugs.  If you come across one, please leave a comment here detailing it.  If there is a feature you’d like to see, please feel free to comment here.  If I get enough of a response, I’ll setup a trac site for the project.
  • The preview only shows you about 10 different (the most common in my opinion) settings.  There are options to customize many more colors, and changing those settings will encode the appropriate lines in the config. files, but you will not be able to see what they look like until you load it into Eclipse.
  • I will eventually add options to change font style (bold, italic, underlining) but as of now there is no support for that and you must go in and manually change the settings afterwards (see colorsets for more information on where in the Eclipse UI you can change certain settings).
  • As of now the generator generates files with a lot of debug messages (messages starting with #NOTE or #WARN).  This is debugging information and doesn’t necessarily mean there will be a problem with your files once you import them.
  • If you somehow wind up with a theme that you don’t like or looks broken, you can always exit eclipse, delete the three files generated and restart eclipse at which point it will revert to the standard styles.
 

Finally, the default theme is based on Zenburn (as defined here).  Eventually, I’d like the generator to allow you to start with any number of predefined themes so you can tweak them as you desire, but that functionality is not built in yet — if its something you’d like to see, please consider adding a comment saying so.

Enjoy.

10 comments so far, add yours

Gigabit LAN Upgrade

Not much time to post, so I’ll post the quick bandwidth test we did between two machines before and after the LAN upgrade:

Gigabit-Performance-Table-Fixed

Some notes:

  • For a bandwidth tester we used iperf, a linux script that has been ported to windows via a cygwyn build.
    • For Linux we just ran “yum install iperf”
    • For Windows we googled “iperf windows”, but here is one binary from UCF.
  • The real path just explains the connection between the two units.
  • The window size was automatically determined by iperf.
  • The 10 second test duration is iperf’s default.

And the same data graphically:

Gigabit-Performance-Chart

As you can see, there is considerable bandwidth improvement, even for the integrated NIC (and the PCI NIC is more than twice as fast as that!)

Leave the first comment

How To: Speed up X Tunneling over SSH

I use PuTTY (actually, I use PuTTY tray) and XWin Server to run Linux GUI applications in Windows.  Basically, I have a headless Linux server running and I connect to it via SSH.  While 90% of the things I do on it are command-line based, I’ve yet to find a real good strategy for developing true linux C/C++ applications from a Windows machine, so I end up running Eclipse on the headless box with the windows tunneled to my Windows system.  (I say true because I know I could use something like MinGW or Cygwin and get 99% of the functionality, but I’m picky about that last 1%).  Also, I know there is a really interesting project underway in the Eclipse community to get CDT to build and debug remotely (Remote Development Tools), but I ran into all kinds of problems getting it to work and it really wasn’t worth the time I was spending on it.  Further, X tunneling over SSH worked great.

I wasn’t satisfied though.

My LAN’s throughput chokepoint was my router, limiting connections to 100Mbps so I went ahead and ordered a Gigabit switch and upgraded my cabling (it was pretty shoddy) and one of my NICs.  Before they come in though, I decided to explore some ways to improve tunneling without hardware changes.

The first thing I looked into was changing SSH cipers.

You can look at the available ciphers in PuTTY by going into the configuration window, and under Connection, clicking SSH.  You should see something like this:

SSH-Cipher-Putty-SSH

You can change the the order that these ciphers are considered simply by clicking Up or Down and rearranging them.  I like to put the one I want at the top, then immediately following it with the “–warn below here–” tag, so I know definitively whether I’m getting my #1 choice.

But how do I know which ciper to choose?

Thats a good question, and I didn’t know either.  I did some extremely quick googling and I didn’t get a very definitive answer in the 2 or 3 pages I landed on, but I did find a nice command to run a benchmark on your local system so you can find out for yourself.  The command was:

openssl speed -engine padlock -evp *ciper* 

Where *ciper* is replaced by the name of the cipher you wish to test.  Executing this command starts an OpenSSL script which runs through the algorithm on locally on your computer, measuring the number of computations it can perform per second.

I extended this command with a simple tweak that allowed me to loop through the script using mulitple different ciphers:

for ENC in aes-256-cbc bf bf-ecb rc4 rc4-40 bf-cfb bf-ofb des3 des ; do openssl speed -engine padlock -evp ${ENC} ; done

After executing this, I walked away for a little (probably takes less than 5 mintutes but I didn’t want to introduce any more variability by messing around during the benchmark) and when I cameback it has maybe 150 lines of output, about 15 for each ciper.

Being the quantitative dude that I am, I immediately threw the data into Excel and plotted the processing throughput (in KBps and given on the last line of each iteration) vs block size (given on the second to last line).

The results were really surprising.  Here’s the table I compiled:

SSH-Cipher-Table

Where the highlighted cells indicate maximum throughput for each column or blocksize.  Its pretty apparent from this that RC4 really flys, but the table above doesn’t do the improvement justice.  Lets look at the data plotted:

Note: The label on the vertical axis incorrectly states the units as KBps when they are, in fact, MBps.

(click for full-size)

(click for full-size)

Wow, that is really a tremendous difference between the two RC4 ciphers and the rest, more often than not doubling the throughput of their alternatives.

So with such a huge difference in performance, certainly the RC4 cipher is the default cipher in PuTTY, right?  Actually, no.  At least in my vanilla PuTTY installations, RC4 (AKA Arcfour in PuTTY) is fairly low on the list, only preferred over DES.

I’m not a cryptographer, and I’m not well versed on the differences between the ciphers certainly not well enough to give any informed opinion on which cipher to choose from a security point of view, but from a sheer speed point of view, it would seem that RC4 would preferred.

Now our network is secured from the WAN side by a firewall on our router which we are trusting to prevent unauthorized access to the LAN side, so for us, encrypting SSH sessions that strictly live on the LAN side with a possibly sub-par cipher was acceptable if it provided the latency improvement that it looked like it would.  Your circumstances may differ and I urge you to consider all of them before you go changing settings.

Given that you’ve considered them, and you’re comfortable, lets go ahead and change the ciper priorities and see if we get any improvement.

Simply by clicking the cipher we want to move, then the Up / Down buttons, we were able to configure our priorities to this:

SSH-Cipher-Putty-SSH-Modified

The idea here being that if, for some reason, we can’t establish a link with RC4, we’ll be notified.  We could then go back into the settings and move the “–warn below here–” line down a line and try again, hoping to connect with the Blowfish ciper, the second quickest cipher or otherwise receive a warning.

Results

Unfortunately, I don’t have any quantitative data to show the latency improvement, although I suppose you could demonstrate a bandwidth / throughput increase by doing a file transfer, however it is abundantly clear that there was, in fact, a huge latency improvement.  Things that caused annoying lag (eg clicking on the scrollbar to scroll quickly) showed a noticeable improvement.

As it is, without real data to show the improvement, you are left to experiment with the settings yourself, and, given you feel comfortable, I strongly encourage doing so — as it really seems to make a difference.

I’ll be sure to check back here when we receive the Gigabit hardware and present a before and after look at our LAN transfer speeds.

One comment so far, add another

Color Schemes for Eclipse CDT – Part 3

As requested, I’ll post some of the theme’s I’ve done.  Once I finalize the configuration utility, I should be able to pump out a ton of these, but until then… here are my two favorite right now.

5 comments so far, add yours