Setup_Venti_Server
This document logs steps to setup a venti server, especially in a d710 machine in Emulab.
1. Install Plan 9 from User Space, following the instructions here http://swtch.com/plan9port/
To install Plan9Port in Ubuntu, you also need to install the following packages.
apt-get install libx11-dev x11proto-core-dev libxt-dev libxext-dev
2. Read introductions about data structures in venti here http://swtch.com/plan9port/man/man8/venti.html
3. Initialise disks based on instructions here http://swtch.com/plan9port/man/man8/venti-fmt.html
Use one of these two 250GB SATA disks in a d710 machine as the storage device. We will use 200GB for data log store, 10GB for index store(5% of data log) and 4GB for blooming filter.
3.1 Create a partition for the data log
Command (m for help): p
Disk /dev/sdb: 250.0 GB, 250000000000 bytes
255 heads, 63 sectors/track, 30394 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x6101c06f
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-30394, default 1): 1
Last cylinder, +cylinders or +size{K,M,G} (1-30394, default 30394): +200GB
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (24317-30394, default 24317):
Using default value 24317
Last cylinder, +cylinders or +size{K,M,G} (24317-30394, default 30394): +10GB
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 3
First cylinder (26750-30394, default 26750):
Using default value 26750
Last cylinder, +cylinders or +size{K,M,G} (26750-30394, default 30394): +4GB
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
3.2 format the disk partition into an arena partition
utos@node:/usr/local/plan9$ sudo ./bin/venti/fmtarenas -a 512M -b 8K arena /dev/sdb1 fmtarenas /dev/sdb1: 373 arenas, 200,005,074,944 bytes storage, 524,288 bytes for index map
3.3 Format the created partition into a venti index section
utos@node:/usr/local/plan9$ sudo ./bin/venti/fmtisect -b 8K index /dev/sdb2 fmtisect /dev/sdb2: 2,442,787 buckets of 215 entries, 524,288 bytes for index map
3.5 We divide the 12GB memory in a d710 machine for the venti server, as suggested here. (Make sure you are using 64-bit Linux. Otherwise, the 12GB memory can not be used at one time.)http://swtch.com/plan9port/man/man8/venti.html
4G for bloom filter, 4G for index cache, 1G for lumps and 3G for block cache.
3.7 Format the bloom filter partition
utos@node:/usr/local/plan9$ sudo ./bin/venti/fmtbloom -n 25M /dev/sdb3 warning: not using entire 4,005,711,360 bytes; using only 536,870,912 bytes warning: using only 128MB - not enough blocks to warrant more fmtbloom: using 128MB, 28 hashes/score, best up to 26,843,545 blocks
4. Setup venti server
4.1 Prepare a configuration file for venti server, based on this page. http://swtch.com/plan9port/man/man8/venti.html
$ cat /tmp/venti.conf index main arenas /dev/sdb1 isect /dev/sdb2 bloom /dev/sdb3 mem 1G bcmem 3G icmem 4G httpaddr tcp!*!8080
4.2 Fmtindex reads the configuration file venti.conf and initializes the index sections to form a usable index structure.
utos@node:/usr/local/plan9$ sudo ./bin/venti/fmtindex /tmp/venti.conf fmtindex: 373 arenas, 1,221,550 index buckets, 199,998,930,944 bytes storage
5. Start the venti server
$ sudo ./bin/venti/venti -c /tmp/venti.conf -d -s 2011/0404 18:28:48 venti: conf...init...icache 0 bytes = 1,000 entries; 4 scache sync...announce tcp!*!venti...serving.
6. Go to http://machinename:8080/storage to see brief storage totals.
7. When you use firefox to connect http server provided by venti, you might find the format provided by venti(8) man is not exactly correct. Especially, /set/variable will report "can not find page" error. When you face this problem, you might consider to use the following format.
Display all variables http://machinename:8080/set
See a single variable http://machinename:8080/set?name=variable
set the value for a variable http://machinename:8080/set?name=variable&value=(0,1)
Tips: When using imageunzip, the output filename should be a partition, like /dev/sdb1. If the output filename is a device, then the partition table for this device will be destroyed even if the slice number is specified. So you should use as the following.
sudo ./imageunzip -o /mnt/images/boss/FBSD47-STD.ndz /dev/sdb1
The following command is not equal to the above command.
sudo ./imageunzip -o -s 0 /mnt/images/boss/FBSD47-STD.ndz /dev/sdb