>/dev/null

Faster mobile browsing through a compressing proxy server

tdomhan — Sun, 25 Jul 2010 20:44:02 +0000

Introduction
A few days ago I tested the Opera web browser on my Android smartphone. The first thing I noticed: It is amazingly fast! This is because all web traffic is routed through an Opera Turbo server, which will compress the complete website and forward it to your smartphone afterwards. You will notice the reduced quality of the images, however it’s worth the increased speed. No doubt this is a great feature. On the other side, Opera had trouble rendering some websites and the (multi-)touch interface is not that great either.

Therefore I wondered if there is any possibility to make use of a similar feature using the standard Android browser. After searching the web for a while, I stumbled upon Ziproxy, which “is a forwarding (non-caching) compressing HTTP proxy server”. It allows you to surf the web blazingly fast while reducing the traffic at the same time. It does so by compressing HTML files and images. This sounded great, so I gave it a try. Please find an install guide below. The first thing I noticed on my desktop computer were the artifacts caused by the reduced image quality. However you will not really see them on your smartphone. In order to use it on an Android smartphone you have to do nothing but to set up a proxy server. (Which is described below)

There are many websites that already compress their hypertexts. Hence the reduced image quality will have the greatest effect on the network traffic. I’ve not had the time to thoroughly test it, yet. I will do so in the next couple of days. Especially in areas with bad network coverage.

Note that Ziproxy is not capable of HTTP persistence. This means a new connection for every element(like images) on a website. This could negatively influence the performance. Consequently I’m assuming that the Opera Turbo is outperforming Ziproxy regarding the rendering speed. Though this does not affect the traffic reduction.

How to install Ziproxy on an Ubuntu server
The first thing you need is a Unix server. There are packages available on the project’s website for many Unix/Linux distributions. If you have an ubuntu version below lucid you have to compile the software yourself. This is how to do so:

Download the tarball from the website, and extract it:
tar xvjf ziproxy-3.1.3.tar.bz2

You have to install some packages in order to be able to compile ziproxy:
apt-get libungif4-dev libungif-bin libpng12-dev libpng12-0 libjasper1 libjasper-dev libjpeg62 libjpeg62-dev zlib1g build-essential

Now you may issue the standard make commands:
./configure make make install

If everything went well you will find the binary in /usr/local/bin/, you may want to check that.

Copy the config file to an appropriate location:
cp -R etc/ziproxy/ /usr/local/etc/

Create a new user:
adduser ziproxy --no-create-home --disabled-login

Now edit the config file, (/usr/local/etc/ziproxy/ziproxy.conf):

RunAsUser = “ziproxy”
RunAsGroup = “ziproxy”
You may want to change the default port.

I recommend to set up some authentication, otherwise your proxy might be abused by someone:
Uncomment the line referring to a “PasswdFile”. This file should contain a username and a plain text password separated by a colon.

Test it:
ziproxy -d -c /usr/local/etc/ziproxy/ziproxy.conf
Your proxy is now running. You have to configure your client to use it. This depends on which browser you are using. Below you will find a description on how to configure your Android smartphone.

Autostart:
Copy the statup script to /etc/init.d:
cp etc/init.d/zipproxy /etc/init.d/

Edit the paths inside this file according to your configuration.

Now add it to some runlevels, e.g. 1 through 5, using a tool like sysv-rc-conf.

How to configure a proxy on an Android smartphone

Install and launch Any Cut
Create a new shortcut -> Activity -> Proxy Settings
Launch the created shortcut.(You’ll find it on some of your home screens)
Enter your server’s domain and the port you specified in the config file
???
Profit

HTC Desire adb driver for windows

tdomhan — Sun, 27 Jun 2010 10:17:28 +0000

If you want to develop android applications on a windows system you have to install a special usb driver, in order to debug directly on the device. When you plug in your device the first time windows will fail to install the appropriate drivers. In the device manager a exclamation point will appear beside the ADB device. ADB stands for Android Debug Bridge. If you are using a nexus one or G1 you must install the usb driver through the Android SDK and AVD manager. Afterwards let windows search the SDK folder for the usb driver. Voilà, that’s it.

However this will fail with a HTC Desire. Windows is not able to use the driver supplied by the Android SDK. In order to debug your application on a htc desire do the following:

Connect your device to your PC.
Go to your phones SD Card.
There is a folder called HTC Sync 2.0.28, in which the installer for HTC Sync resides.
Install HTC Sync.
Ok. That’s it. You now got some crappy HTC sync software installed. However as a sideeffect the driver for the debug bridge is installed, too.

springer ebook crawler

tdomhan — Wed, 18 Nov 2009 16:39:50 +0000

Springer ebooks are divided up into many small pdf files. Downloading them one at a time sucks. I wrote a little perl script that downloads purchased ebooks from the Springer website and concatenates them. It can be downloaded here.

creating images of encrypted partitions with partimage

admin — Wed, 28 Oct 2009 16:59:36 +0000

partimage is a tool for creating images of whole partitions. It supports all major file systems, like ext2/ext3. It will only backup the used blocks of a partition and thus save space. In order to do so, it needs to detect the type of the file system. Encrypted file systems can not be used directly. You first have to open the device:

cryptsetup luksOpen /dev/sdXX SOMENAME

After that you can use the new device /dev/mapper/SOMENAME. You have to specify this device directly on the command line, because it will not be auto detected by partimage in the ncurses mode. Another important option is -M, which will not backup the MBR. If you forget this parameter the whole thing will fail. In the end the command looks like that:

partimage -z1 -d -M save /dev/mapper/SOMENAME FILENAME.img.gz

new Ubuntu version approaching

tdomhan — Thu, 22 Oct 2009 20:39:16 +0000

In exactly one week from now a new version of Ubuntu will be released. The new features can be found in the release notes.

ubuntu/debian raid10 installation

tdomhan — Mon, 19 Oct 2009 21:31:47 +0000

I’m currently setting up a new file server. My first plan was to use Ubuntu 8.04 LTS. The system has four sata drives, that should be combined to a raid10. Actually I wanted to use two raid1 which are combined into a striped logical volume via LVM. The server has no cd/dvd drive. Therefor an installation via PXE is the way to go. The text installer is capable of creation md raids and LVM volumes. Though I didn’t find any possibility to create a striped volume, neither in the debian nor in the ubuntu installer.

The solution is the following:
1. create the md raids and the LVM volumes outside the installer, with some live system.(CD/DVD or USB)
2. start the PXE installer
3. in the partition editor choose “manual setup”(or something similar)
4. the system will detect the created volumes which can be further used

This is only possibile with the current stable Debian installer, not with the Ubuntu 8.04 LTS installer. I don’t know if the situation change in a newer Ubuntu version. It probably did as they take the code from Debian. Whatever…so my server will be a Debian system.

software raid10 under linux

admin — Sat, 17 Oct 2009 22:14:05 +0000

There are two different ways to build a raid10 software raid under linux. The first just uses md to build a raid10 and layer a Logical Volume Manager (LVM) ontop. The LVM is not necessarily needed, but it offers useful features like snapshots. The other approach involves creating two raid1 arrays with md and combining those to a stripe inside the LVM. I can’t tell which is technically the better solution. I read somewhere that the pure raid10 setup had a better performance. In order to find out I did a short benchmark with bonnnie++ on a machine with four sata drives. The results can be seen here:

		Sequential Output						Sequential Input				Random Seeks			Sequential Create						Random Create
	Size:Chunk Size	Per Char		Block		Rewrite		Per Char		Block		Random Seeks		Num Files	Create		Read		Delete		Create		Read		Delete
		K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	/ sec	% CPU		/ sec	% CPU	/ sec	% CPU	/ sec	% CPU	/ sec	% CPU	/ sec	% CPU	/ sec	% CPU
2x RAID1 + LVM	1488M	12595	97	81768	92	67659	57	14597	98	83011	17	372.4	3	256	10707	68	6156	15	4382	25	8580	56	3795	10	2912	18
RAID10 + LVM	1488M	12599	97	80165	88	35164	29	13970	94	55343	13	371.5	2	256	12129	76	5738	13	4061	23	9966	63	3217	9	3141	19

The benchmark shows, that there is not that big of a difference between those two.

But there is one point that made my decision. It is currently not possible to resize a raid10. Though is is possible to resize the two raid1 arrays individually.

Android tethering without root access

tdomhan — Tue, 29 Sep 2009 20:46:13 +0000

I stumbled upon a nice piece of software called Azilink. It allows you to use your Android phone to connect to the Internet. You first need to create an connection to the phone with the help of the Android debugger. On the phone Azilink provides an openvpn server. Next you have to start the openvpn client software in order to connect to it. Furthermore the IP address of an DNS server is needed. Detailed instructions can be found on the google code project page: http://code.google.com/p/azilink/

Android SDK under Ubuntu 9.04

tdomhan — Tue, 01 Sep 2009 20:52:44 +0000

In order to connect to your usb connected Android phone from you Android SDK under Ubuntu 9.04, you have to do the following:

1. Login as root and create the file:

 /etc/udev/rules.d/51-android.rules

The name is important. If you replace the 51 with a 50 all will fail!

2. Copy the following into the file:

SUBSYSTEM=="usb", SYSFS{idVendor}=="0bb4", MODE="0666"

3. restart udev

sudo /etc/init.d/udev reload

4.Plugin you debugging enabled Android phone

5. inside your SDK folder run:

tools/adb devices

if everything worked fine you phone should be listed.
6. enjoy!

Win Vista -> Win 7 upgrade

tdomhan — Sun, 23 Aug 2009 12:03:52 +0000

Yesterday I upgraded from Win Vista to Win 7. The process itself takes quite a while, but is quite smooth. All your settings and and programs will be preserved. (Doing a backup before upgrading is recommended, though)
My windows partition was quite full before the upgrade. You need 16GB of free space for the upgrade. It was quite a struggle to free so much space. To my suprise after the upgrade 33GB of 70GB were free. So Win 7 seams to need less HD space. All drivers work after the upgrade. I didn’t have to install any new. Well exept one.. the tap driver. It is needed to create virtual network interface cards. This might render openvpn unsable. We’ll see in october when I need openvpn to access the internet inside the university. But I will probably use my Ubuntu there…so who cares?
Ok there is another problem concerning the ext drivers, which allow me to access partitions of my linux installation(and my external HD). You can install them in windows vista compatibilty mode. But upon reboot the partitions will be gone and you have to reinstall the drivers. This could get quite annoying. But well… I’m planning to build a fileserver, so that windows can access my external HD via SMB. To sum up:

smooth upgrade procedure from win vista to win 7
win7 needs less HD space
win tap drivers not working
ext drivers not fully working