codaファイルシステムを使ってみる
分散ファイルシステムにはたくさんの種類があるが、マイナーどころということでこれまで何度も挑戦して失敗してたcodaファイルシステムにもう一度挑戦してみた。本家はここにあり、ドキュメントやソースが入手できる。ただし、ドキュメントは非常に古いので注意(netbsd-1.3に対応とかいってるし)。
インストール
codaはクライアント/サーバシステムでサーバは複数のノードでクラスタのようになって動くらしい。現在のサポート状況は、
サーバ: linux, FreeBSD, NetBSD
クライアント: linux, FreeBSD, NetBSD, WindowsXP
とのこと。この中でNetBSDで挑戦してみた。
NetBSDではpkgsrc対応しているためnet/codaから
% sudo make install
でインストールできる。ただ、一部のperlスクリプトが/usr/bin/perlを要求するため、/usr/bin/perlへsymbolic linkを張っておくとよい。以上でサーバとクライアントの両方がインストールされる。
実験ということで、サーバとクライアントが同居する状態で設定を行う。
サーバ設定
便利な設定スクリプトvice-setupがついているのでそれを使う。codaの世界ではサーバはvice, クライアントはvenusと呼ばれているようだ。
ruby: {73} sudo vice-setup
Welcome to the Coda Server Setup script!
What is the domain name for your servers? arara.rim.or.jp
You already have a file /usr/pkg/etc/coda/server.conf!
Continueing will remove that file.
Do you want to continue? [yes/no] yes
Setting up config files for a coda server.
Do you want the file /usr/pkg/etc/coda/server.conf created? [yes] yes
What is the root directory for your coda server(s)? [/vice] /dsk/vice
Setting up /dsk/vice.
Directories under /dsk/vice are set up.ここまでで、サーバ内の基本的な設定ができたことになる。ドメイン名は重要でresolv(3)が出来ることが必要。
Is this the master server, aka the SCM machine? (y/n) y
SCMというのは何の略が分からないが、要するにマスターサーバとなるかどうかを指定する。今回は一台なので当然yを選択。すると各種認証に使うトークンを設定することになる。ここで3種類のランダムトークンを設定するが、すべて8文字であることが必要。サーバIDを指定するとSCMサーバおよび認証サーバとしての設定が終わることになる。
Setting up tokens for authentication. The following token must be identical on all servers. Enter a random token for update authentication : elephant The following token must be identical on all servers. Enter a random token for auth2 authentication : 12345678 The following token must be identical on all servers. Enter a random token for volutil authentication : abcdefgh tokens done! Setting up the file list for update client Filelist for update ready. Do you want to start the server at boot time? (y/n) y Startup scripts now installed. Now installing files specific to the SCM... Setting up servers file. Enter an id for the SCM server. (hostname ruby.arara.rim.or.jp) The serverid is a unique number between 0 and 255. You should avoid 0, 127, and 255. serverid: 1 done!
続いてアドミンユーザとグループの設定をする。パスワードは'changeme'と固定だ。後で変える。
Setting up users and groups for Coda You need to give me a uid (not 0 or 1) and username (not root) for a Coda System:Administrator member on this server, (sort of a Coda super user) I will create the initial administrative user with Coda password "changeme". This user/password is only for authenticating with Coda and not for logging into your system (i.e. we don't use /etc/passwd authentication for Coda) Enter the uid of this user: 1000 Enter the username of this user: codaadmin
codaはRVMを使っていてその裏打ちとなるストレージエリアを設定する。rawパーティションが望ましいが実験であるため、通常のファイルを指定した。ここではlogに20M, dataに128Mを指定した。
A server needs a small log file or disk partition, preferrably on a disk by itself. It also needs a metadata file or partition of approx 4% of your filespace. Raw partitions have advantages because we can write to the disk faster, but we have to load a copy of the complete RVM data partition into memory. With files we can use a private mmap, which reduces memory pressure and speeds up server startup by several orders of magnitude. Servers with a smaller dataset but heavy write activity will probably benefit from partitions. Mostly read-only servers with a large dataset will definitely benefit from an RVM data file. Nobody has really measured where the breakeven point is, so I cannot really give any hard numbers. ------------------------------------------------------- WARNING: you are going to play with your partitions now. verify all answers you give. ------------------------------------------------------- WARNING: these choices are not easy to change once you are up and running. Are you ready to set up RVM? [yes/no] yes What will be your log file (or partition)? /dsk/vice/rvm.log The log size must be smaller than the available space in the log partition. A smaller log will be quicker to commit, but the log needs to be large enough to handle the largest transaction. A larger log also allows for better optimizations. We recommend to keep the log under 30M log size, many people have successfully used as little as 2M, and 20M has worked well with our servers. What is your log size? (enter as e.g. '20M') 20M Where is your data file (or partition)? /dsk/vice/rvm.data The amount of RVM we need to store the metadata for a given amount file space can vary enormously. If your typical data set consists of many small files, you definitely need more RVM, but if you tend to store large files (mp3s, videos or image data) we don't need all that much RVM. Here are some random samples, mp3 files ~0.08MB RVM per GB. jpeg images ~0.50MB RVM per GB. email folders ~37.8MB RVM per GB (maildir, 1 file per message) netbsd-pkgsrc ~180MB RVM per GB (large tree but not much data) To get a more precize number for your dataset there is a small tool (rvmsizer) which can reasonably predict the amount of RVM data we need for a file tree. Remember that RVM data will have to be mmapped or loaded into memory, so if anything fails with an error like RVM_EINTERNAL you might have to add more swap space. What is the size of you data file (or partition) [32M, 64M, 128M, 256M, 512M, 768M, 1G]: 128M -------------------------------------------------------- WARNING: DATA and LOG partitions are about to be wiped. -------------------------------------------------------- --- log area: /dsk/vice/rvm.log, size 20M. --- data area: /dsk/vice/rvm.data, size 128 MB. Proceed, and wipe out old data? [y/n] y LOG file has been initialized! Rdsinit will initialize data and log. This takes a while. rvm_initialize succeeded. Going to initialize data file to zero, could take awhile. done. rds_zap_heap completed successfully. rvm_terminate succeeded. RVM setup is done!
最後にファイルシステムのブロックを格納するエリアを設定する。実際のデータがここに格納される。
ファイル数は256K個にした。
Directories on the server will be used to store container files that hold the actual data of files stored in Coda. Directory contents as well as metadata will be stored in the RVM segment that we already configured earlier. You should only have one container file hierarchy for each disk partition, otherwise the server will generate incorrect estimates about the actual amount of exportable disk space. Where shall we store your file data [/vicepa]? /dsk/vice/vicepa Shall I set up a vicetab entry for /dsk/vice/vicepa (y/n) y Select the maximum number of files for the server. [256K, 1M, 2M, 16M]: 256K Server directory /dsk/vice/vicepa is set up! Congratulations: your configuration is ready...
長い道のりだったがこれが最後。スタートしてみるといろいろログが出て、こんな風になるとOK。
Shall I try to get things started? (y/n) y - Coda authentication server (auth2 &) - Coda update server (updatesrv) - Coda update client (updateclnt -h ruby.arara.rim.or.jp) Creating /dsk/vice/spool - Coda file server (startserver) Nice, it looks like everything went ok Now I'll try to create an initial root volume - createvol_rep / ruby.arara.rim.or.jp/dsk/vice/vicepa Replicated volumeid is 7f000000 creating volume /.0 on ruby.arara.rim.or.jp (partition /dsk/vice/vicepa) V_BindToServer: binding to host ruby.arara.rim.or.jp V_BindToServer: binding to host ruby.arara.rim.or.jp Set Log parameters Fetching volume lists from servers: V_BindToServer: binding to host ruby.arara.rim.or.jp GetVolumeList finished successfully ruby.arara.rim.or.jp - success V_BindToServer: binding to host ruby VLDB completed. <echo / 7f000000 1 01000001 0 0 0 0 0 0 0 >> /dsk/vice/db/VRList.new> V_BindToServer: binding to host ruby VRDB completed. Do you wish this volume to be Backed Up (y/n)? [n] That seems to have worked... If you have a working Coda client you should now be able to access the new Coda realm - cfs lv /coda/ruby.arara.rim.or.jp/ enjoy Coda. for more information see http://www.coda.cs.cmu.edu.
クライアントの設定
こんどはクライアントの設定を行う。サーバと同じくvenus-setupなるスクリプトで行う。venus-setupコマンドでデフォルトのサーバを設定するだけ。
ruby: {4} sudo venus-setup ruby.arara.rim.or.jp
接続
クライアントの起動を行う。
ruby: {6} sudo venus
Date: Sun 12/30/2007
21:55:04 Coda Venus, version 6.9.2
21:55:04 /dsk/coda/LOG size is 214880 bytes
21:55:04 /dsk/coda/DATA size is 859520 bytes
21:55:04 Initializing RVM data...
21:55:04 ...done
21:55:04 Loading RVM data
21:55:05 Starting RealmDB scan
21:55:06 Found 1 realms
21:55:06 starting VDB scan
21:55:06 0 volume replicas
21:55:06 0 replicated volumes
21:55:06 0 CML entries allocated
21:55:06 0 CML entries on free-list
21:55:06 starting FSDB scan (85, 100000) (25, 75, 4)
21:55:06 0 cache files in table (0 blocks)
21:55:06 85 cache files on free-list
21:55:06 starting HDB scan
21:55:06 0 hdb entries in table
21:55:06 0 hdb entries on free-list
21:55:06 Mounting root volume...
21:55:06 /coda now mounted.
21:55:06 Venus starting...
ログインしてみる
ログインするにはclogコマンドを使う。パスワードは例の'changeme'だ。
ruby: {7} ls -l /coda
ruby: {8} clog codaadmin@ruby.arara.rim.or.jp
username: codaadmin@ruby.arara.rim.or.jp
Password:
ruby: {9} ls -l /coda
total 4
drwxr-xr-x 1 root 65534 2048 Dec 30 20:51 ruby.arara.rim.or.jp
ruby: {16} cfs la /coda/ruby.arara.rim.or.jp/
System:Administrators rlidwka cfsコマンドはACLなどを設定できる便利コマンドで、System:Administratorsというのは管理者グループである。
rlidwkaというのはパーミッションの記号名である。詳しくはman cfsして欲しい。
書き込み
ここまでくれば書き込み可能
ruby: {17} echo a > /coda/ruby.arara.rim.or.jp/tt
ruby: {20} ls -al /coda/ruby.arara.rim.or.jp/
total 10
drwxr-xr-x 1 root 65534 2048 Dec 30 22:06 .
dr-xr-xr-x 1 root 65534 2048 Dec 30 21:55 ..
-rw-r--r-- 1 admin 65534 2 Dec 30 22:03 tt
ruby: {21} ctokens
Tokens held by the Cache Manager for k-1:
@ruby.arara.rim.or.jp
Coda user id: 1000
Expiration time: Mon Dec 31 22:55:32 2007ctokensコマンドで誰でログインしているか、チケットの有効期限が表示される。ここではcoda user idが1000つまりcodaadminであり、大晦日まで有効だというのが分かる。
