ACEDB Server/Client system

(Acedb 4.8 Socket version)


Preface

You should note that in 4_7 the RPC based versions of the client server were called "aceserver" and "aceclient", these names have been changed for the socket versions to "saceserver" and "saceclient". The RPC and socket programs cannot be used interchangeably: you must use the saceclient, not the aceclient, with the saceserver and vice versa.

Introduction

Acedb consists of a collection of programs that either access the database files directly (e.g. tace, xace) or get access to the database by issuing commands across a network to a server program (e.g. saceserver/saceclient). This latter approach has several advantages:

Basis of client/server communication

Textual queries:

It's useful to understand just what information passes between the client and server. There is an implicit layering in all of the acedb programs in that they all provide an interface whereby queries in plain text form, e.g.

		"find model"
		"list"
		etc.
can be passed to "interpreter" code which will then call the low level database routines required to fulfill those commands. It is at this level that the split between client and server is made. The client reads commands typed in by the user (or perhaps read from a file containing a collection of commands that the user has supplied), and then passes these commands across the network to the server. The server contains the "interpreter" code which scans the commands and makes calls to the low level database routines to access the actual database. The server then constructs a reply as plain text which it sends back to the client, the client displays this reply to the user. In this way there is no difference really between tace and saceclient, both are command line based, it's just that saceclient accesses the database via a server across the network.

Graphical displays - an aside

Graphical display is more difficult because the graphical code, for performance reasons, must itself call low level routines to access information in the database. There are however currently two methods of graphical display:

These interfaces will not be demonstrated here, the former because it is experimental, the latter because it is quite complex to set up and beyond the scope of this course.

Things you need to understand to use the server/clients

Server Instances, Port numbers and databases

If you are familiar with Unix/computers then skip this paragraph, otherwise..... We tend to think of a program as the single file on the disk that contains that program. It is more correct however to think of "instances" of a running program: many copies of the same program can be running at the same time. This is important when thinking about the server because each time a server is started up it can only access one database. Hence to allow access to many databases on a single machine, there may be many instances of the server program running at the same time, each accessing a different database.

Now, which instance of the server do you contact to access a particular database ? To understand this we need to know a bit about port numbers.

Computers make sure that the many different types of communications they receive over the network go to the correct programs by associating each program with a unique "port" number, e.g. if you look in the file /etc/services you can see a list of port numbers/programs, some of which are essential operating system services. For now all you need to know is two things:

  1. port numbers below 1024 are reserved for operating system use and you should not try to use them (the operating system will probably prevent you from using one anyway), otherwise you can use any number that is not mentioned in /etc/services.
  2. When we start up the server we tell it what database we want to access AND which port number it is to use.

You can probably already see that the port number is like a "key" to the database. When the client contacts a server on a particular port number, the server on the other end of that port will be accessing a particular database. To access a different database we could start up another instance of a server running on a different port and using a different database.

So when you start up the client you do not say which database you want to access, instead you specify the machine you want to contact and which port on the machine you want to connect to.

(NOTE that the client and server can be on the same or different machines, the commands are the same.)

An example:

(In this and other examples "YOURDB" stands for whatever directory contains database, e.g. /home/johnsmith/mydatabase, so that if you see "YOURDB/wspec" it means /home/johnsmith/mydatabase/wspec)

Let's say we are working on the machine testbed and start up the server like this:


> saceserver YOURDB 2000

then the client can be started up like this:


> saceclient testbed -port 2000

The client will now be able to access YOURDB via the server on testbed on port 2000. Note how the client needs to be told the machine and port number and then the database that will be accessed is implicit from this because of how the server was started up.

Trying out the server/client code

Unfortunately controlling access to files/programs on computer systems is not straight forward, it usually consists of the interaction of several different components/facilities that must all be set correctly to achieve the desired level of security. In our case we will need to consider the following:

The first two items are about providing basic UNIX protection, the last item is about providing control via the servers access mechanisms. We will consider these two separately.

You need to understand Unix userids/groups and file permissions for this section.

Step 1) - server/client program permissions:

We can use Unix permissions to limit who can run the server and client programs. This is an important first line of defence in setting up a database securely.

Usually it is sensible to limit who can run the server to a few administrators or even to a single special administrator id. For instance, assuming that we have a special administrator id called elegans who should be able to run the server, but that we also want a group of chosen userids (e.g. fred and michele) to be able to run the server program for testing then we need to do the following:

This means that only elegans and users in the admingroup will be able to run the saceserver and replace it with a new version or look at it with an editor (in this case the latter is unlikely because the file is compiled code).

The client could be similarly limited to a few users but this can be more trouble than its worth because the group for the client program file would have to be constantly changed as users came and went. A more flexible approach is to provide general access to the client to run it (but not look at it with an editor or replace it):


> ls -l saceclient
-rwxrwx--x   1 elegans   admingroup      301600 Jan 18 20:32 saceclient

The means that elegans and admingroup users will be able to run, replace and look at the client, anybody else can run the client. In the following sections we will see how, even though anyone can run the client, we can control who actually gets access to the database via the servers access control mechanism.

Step 2) - database file permissions:

We must set the file permissions of the database itself. Its not much good setting the permissions of the server program but leaving the database files so that anyone can alter them manually !

Given the preceding permissions/groups it is sensible to set the file permissions of the database to the same as the server program. This will allow the server program to access the database and also the elegans userid and admingroup userids. The files/directories of the database should be set so that they give read, write and execute permission to elegans and admingroup and nothing to anyone else:


> ls -l YOURDB

drwxrwx---   5 elegans   admingroup        1024 Apr 21  1999 database/
drwxrwx---   3 elegans   admingroup        3072 Nov  4  1998 whelp/
drwxrwx---   2 elegans   admingroup        1024 Jan 26  1999 wspec/

               etc. etc.

The easiest way to do this is to "cd" to the parent directory of the database and then use "chown", "chgrp" & "chmod" commands with the "-R" flag to change the ownership and permissions of the of the database directory itself and all its sub-directories and files:


> cd 
> chown -R elegans ./YOURDB
> chgrp -R admingroup ./YOURDB
> chmod -R ug+rwx ./YOURDB
> chmod -R o-rwx ./YOURDB

We now have the server and client program permissions set up and the database permissions set, now we need to set up the servers access control system.

Step 3) - Server Access to the database:

All acedb programs (xace, tace, saceserver etc.) use the file YOURDB/wspec/passwd.wrm to make sure that the userid running the program has permission to write to the database. If the program can find the userid in this file then the program will be able to update the database, otherwise the database will be read-only.

If you run the server in the foreground as we did in the class then the server will be running as though it were you, so your userid must be in passwd.wrm. But what if the server is started up automatically by the operating system via inetd ? In this case the userid should be the one that you put in the file /etc/inetd.conf (see the later section on inetd), e.g. if the inetd.conf entry is:


allgenes stream tcp wait joebloggs /home/me/saceserver (contd..)
                              saceserver /home/mydatabase
then you should put joebloggs in YOURDB/wspec/passwd.wrm. Note that if you forget, you cannot do this while the server is running, you must stop and restart it.

So passwd.wrm should contain the userids of anyone who is allowed to start the server. These entries allow the server program itself to get access to the database, the next section shows how to give users access to the database.

Step 4) Global user permissions:

At this stage we will use the simplest option and make the database readable by anybody so that we can get on and try out the server/client. To do this you need to edit the server configuration file YOURDB/wspec/serverconfig.wrm. This file contains the keywords "READ" and "WRITE", these control read and write access permissions for all users. (These options will be discussed in more detail in the following userid/password section.)

To allow anyone to have read access you need to edit the line that contains "READ NONE" so that it looks like this:


READ WORLD

This tells the server to let anyone have read access to the database.

Step 5) Starting the server:

The simplest way to start the server is from the command line in an xterm, this is what we will do. Make sure you have at least two xterms running on your X Windows screen. Decide what port number you want to use (must be greater than 1023 and not in /etc/services) and then start the server by typing:


> saceserver  /your/database/directory  your_port_number

If the server has started correctly then the xterm prompt will not be redisplayed, the xterm will "hang" until the server program exits.

Step 5) Starting the client:

You now need to start the client in a separate xterm window. Since you have started the server on your own machine you now need to know what the name of your machine is. You can find this out by typing the command "hostname":


> hostname

this should return the name by which your machine is known on the network.

Now start the client by typing:


> saceclient your_machine_name -port your_port_number

The client will prompt you for a userid and passwd:


Please enter userid: XXXXXXX
Please enter passwd: 
Because you have set READ permissions to WORLD you can just enter any word you like for userid and any word you like for passwd (but you must enter something), this is like ftp when you use it with the "anonymous" login id.

If you are successful then you should see output like this:


acedb@your_machine_name>

you can now type commands in at the "acedb" prompt to explore the database.

Trying out the server/client

tace and saceclient share the same command interface which enables you to interact with the database, do the following to get an idea of the scope of the interface:


acedb@your_machine_name> ?
- will give you a list of all possible commands

acedb@your_machine_name> help
- will give you a brief overview of the sets of commands and what they do

acedb@your_machine_name> help topic
- will give you more detailed help on a particular topic/command

Try out a few commands:


acedb@your_machine_name> classes
- this gives a list of all the visible class names and how many objects they contain

To see the model or schema for a class try the "model" command:


acedb@your_machine_name> model oligo
- will show the model for the oligo class, you should see something like:

?Oligo  Sequence UNIQUE Text    // verbatim sequence - useful
        In_sequence ?Sequence XREF Oligo
        STS STS1 UNIQUE ?STS XREF Oligo1
            STS2 UNIQUE ?STS XREF Oligo2

When you want to stop the client simply type:


acedb@your_machine_name> quit

You could try getting access from another machine by using an xterm on someone elses machine and once again issuing the command


> saceclient your_machine_name -port your_port_number

Note how the same command gets you access to the server from a completely different machine. All that is needed are the machine name and port number. Provided your network security system allowed it, you could even use your server from a machine in a different country although you would need to use the long form of the machine name, e.g. your_machine_name.hgmp.ac.uk or whatever.

To stop the server, go to the xterm running the server and press and hold down the 'Ctrl' key and press the 'c' key. This will send a special signal to the server program that will cause it to exit (later you will learn of a better way to stop the server).

Summary

To recap, you have learnt the basic set up of the database and server/client programs, how to start the programs, basic use of the client and how to stop the programs. The following sections will discuss in more detail how to control access to a database, how to shut it down properly, how to monitor the server, and how to tell the operating system to start up the server automatically via inetd every time a client connects to the port for your database.

Getting write access

So far you have only given read access to clients signing on. In fact it is possible to give global write permission in the same way as was given for READ permission: edit the file YOURDB/wspec/serverconfig.wrm and edit the line that contains "WRITE NONE" so that it looks like this:


WRITE WORLD

This tells the server to let anyone have write access to the database (!). Needless to say this is probably not what you would want to do in the real world.

Start the server as before:


> saceserver  /your/database/directory  your_port_number

Now start the client:


> saceclient your_machine_name -port your_port_number

To find out if you have write access, type in the "save" command:


acedb@your_machine_name> save

If you have write access you will see a message like this:


// Response: 24 bytes.
// 0 Active Objects

If not, you will see a message explaining that you do not have write access:


// Reponse: 64 bytes.
// Sorry, you do not have Write Access.
// 0 Active Objects

If you did not get write access then check through each step carefully and try again.

Access control is covered in more detail in the next section.

Controlling database access more rigorously

Overview

Access to the database is controlled by a combination of the READ/WRITE keywords in YOURDB/wspec/serverconfig.wrm and userids/groups in YOURDB/wspec/serverpasswd.wrm. The READ/WRITE keywords control the overall access to the database for reading and writing and the entries in the serverpasswd.wrm file control individual access to the database.

Userids and Passwords

The server uses a combination of the READ/WRITE keywords introduced in the previous section and userids/passwords to control access to the database by users. userids/passwords work in the same way that they do for Unix or other operating systems.

Global read/write permissions

The valid combinations of READ & WRITE keywords are:


					READ
			NONE  		PASSWD		WORLD

	  NONE	        OK              OK		OK

WRITE	PASSWD		OK		OK		OK

	 WORLD		no		no		OK

Note that if every user has write permission because WRITE is set to WORLD, then there is no point in setting READ to NONE or PASSWD, READ will be redundant and is treated as WORLD in these circumstances.

Note also that you can set both READ and WRITE to NONE because this still allows admin users to access the database.

Userids/Passwords

There are three classes or groups of database users: read, write or admin and they have the following privileges:

read
can read but not write to the database and not use any admin commands.
write
can read/write to the database but not use admin commands.
admin
can read/write and use admin commands.

If users are required to use a password to access the database then they must belong to one of these groups and their userid/password must be stored in the database.

Setting up users/passwords

Once an admin user has been set up then this user can set up new users with passwords and assign the user to a group, its also possible to do this by manually editting the file, and indeed this is necessary to create the first admin user. There is a utility program called "makeUserPasswd" which can be used to make valid user/password entries which can be cut/pasted into the serverpasswd.wrm file:


[edgrif]: makeUserPasswd johndoe
// Please enter passwd: 
// Please re-enter passwd: 
// The following line is a valid entry for wspec/serverpasswd.wrm

johndoe f11dba547a72dda3616da585ab619eb7

[edgrif]: 

You just need to cut/paste "johndoe f11dba547a72dda3616da585ab619eb7" into a new line in the serverpasswd.wrm file and then type "johndoe" on one of the three group lines. If "johndoe" was to be an administrator then serverpasswd.wrm would have the following entries (any line starting with "//" is a comment line):


//
// group entries
//
admin: johndoe
write: some_user another_user
read: a_read_user

//
// user entries
//
johndoe f11dba547a72dda3616da585ab619eb7
some_user 0daa8975f252ebd965391bd3bb5b607b
another_user a910522a4926dbfe91c96a6cb1158654
a_read_user f28855ad9acbb3d0acfd796674c6a8bb

Note the format of the file with each user/passwd being on a separate line and each user being present in just one of the read/write/admin group lines.

The overall picture

read and write user access is affected by the READ/WRITE keywords respectively in YOURDB/wspec/serverconfig.wrm, the following table summarises the relationship:


                        serverpasswd.wrm
serverconfig.wrm        entry in read group ?
----------------        ---------------------

READ NONE               Irrelevant, no one can have read access

READ PASSWD             Must have entry to get read access

READ WORLD              Irrelevant, anyone can have read access


                        serverpasswd.wrm
serverconfig.wrm        entry in write group ?
----------------        ----------------------

WRITE NONE              Irrelevant, no one can have write access

WRITE PASSWD            Must have entry to get write access

WRITE WORLD             Irrelevant, anyone can have write access

You can see from this that you set up databases to be "read only" or "password access only" and so on. The serverpasswd.wrm entry only gets used if either the READ or WRITE keyword in serverconfig.wrm is set to PASSWD.

admin users are different in that they must always access the database using a password, it is not possible to set the database so that "anyone" or "no one" can be an administrator.

Shutting down the server

In a previous example we used Cntl-C to send a signal to the server which forced it to exit. This should only be used as a last resort if you are running the server on an important database. It is much better to use the "shutdown" command which will cause the server to refuse any new client connections and close down the database in an orderly way once the last client has quit. Only users in the admin group are allowed to do this, you could try the command out now that you know how to set up admin users.

Monitoring the server

There are two files used by the server to log errors/information:


YOURDB/database/log.wrm
YOURDB/database/server.log

log.wrm is used by all acedb programs that access the database to record information and errors. If your program crashes then the cause of the crash will be recorded in this file unless the crash was so bad that the program was forced to exit immediately. xace, tace and the saceserver all log errors in this file.

server.log is used by the server to record useful information that may help with debugging or show how many clients have logged on and so on.

You should look in these files if you have problems such as the server not starting, not responding normally or crashing.

It is also useful to use the Unix "ps" command to look at what the server process is doing, assuming the server program is still called saceserver:


> ps -l

will display something like this:


   F S  UID   PID  PPID    C PRI  NI  ADDR   SZ WCHAN  TTY      TIME CMD

8001 S 9767 17388 17337  1.2  44   0     0 7.3M event  ttyp8 0:00.35 saceserver
                  < lots more process entries >

The most useful fields are the SZ and WCHAN fields:

SZ will tell you how much memory the server is using, if you suspect that
   the server is running away and eating up more and more memory it will be
   obvious here (the saceserver is 7.3MB in the above example).

WCHAN gives you an idea of what the server process is doing, in the above
   example it is waiting for an "event", i.e. the server is effectively
   asleep and waiting for a client to connect.

You could try out this command before, during and after a client connection.

"top" is another useful command if your system supports it. It will show you a dynamic display of the top users of CPU on your system. If you suspect the server is looping out of control you can use this command to check this. In the below example you can see that the saceserver is not currently active.


> top

will display something like this:


-----------------------------------------------------------------------
load averages:  0.19,  0.08,  0.07    
109 processes: 1 running, 35 sleeping, 73 idle
CPU states:  7.5% user,  0.0% nice,  8.2% system, 84.2% idle
Memory: Real: 298M/494M act/tot  Virtual: 38M/852M use/tot  Free: 33M

  PID USERNAME PRI NICE  SIZE   RES STATE   TIME    CPU COMMAND
16721 lmc       44    0   33M   19M sleep   0:52  3.10% real-netscape
  640 root      42   -2   16M 9928K sleep   2:47  0.40% Xdec
17389 edgrif    44    0 4576K  745K sleep   0:00  0.20% saceserver
                    etc. etc.

Inetd and the server

So far we have run the server by starting it from an xterm, this is fine for testing but not satisfactory for providing a database service because we would like users to have access to the database whether we are logged on to the machine or not. For this we would like to have server startup automated and fortunately on Unix systems this can be done via an operating system service called inetd (which is short for "Internet Daemon"). inetd is a program that will start up server programs such as the saceserver when a client request arrives at the servers machine. Briefly it does this by listening on all the ports that these servers would normally use and then when a request comes in on a port it starts up the server that should service that port and lets it deal with the request. This section shows what you need to do to have inetd start up the saceserver.

**** TAKE CARE ****

Because you will either need root access on your machine or you will need to get your system support people to do this, we will not actually edit the required files. If you want to try this out you should make sure that you do this with the help of someone who knows how to safely edit operating system files, it is possible to leave your system in a state where it may not reboot if you do not alter these files correctly.

**** TAKE CARE ****

For this example we will assume the following:

our machine name is:                       redbox
our server program is the file:            /home/me/saceserver
the database we want to access is:         /home/mydatabase
the port we are going to use is:           2001
the nickname for this server/database is:  allgenes
the owner of the server program is:        joebloggs
(you will see these bits of information highlighted in the examples below)

We will need to alter two files in the /etc directory which is where the configuration files for most Unix operating systems are kept.

The first step is to tell the operating system which port you want to use, what protocol to use and what symbolic name we want to give to our service, to do this we need to add the following line to the bottom of /etc/services:


allgenes    2001/tcp

Then we need to tell inetd which program to start up when a request comes in on the port we have named


allgenes stream tcp wait joebloggs /home/me/saceserver (contd..)
                              saceserver /home/mydatabase

Any bits that are not highlighted will be the same for any saceserver entry in these files, remember that you will need a new set of entries for each new database you want to make available via a server.

What port numbers should I choose:

    0 to  1023: are reserved for operating system use. DO NOT USE THESE.

 1024 to 49151: are "registered" ports. If you setting up the server to
                be run by inetd you should use one of these (you will need
                to check in /etc/services that your number does not clash
                with one already there)

49152 to 65535: are for casual/ephemeral use. You should use one of these
                if you are just trying out the server from an xterm

Accessing the inetd started server:

Now you should be able to access the server in the same way as before by typing in:

> saceclient  redbox -port  2001

If you have trouble then make sure that you can start the server in the foreground from an xterm and connect to it from a client successfully before attempting the inetd setup.

Documentation

There are further configuration options for the server that can be specified in YOURDB/wspec/serverconfig.wrm, these and further details of the password encryption scheme, along with more server setup information can be found in:


http://www.acedb.org/Development/wdoc/SOCKET_aceserver.html

Summary

You have seen how setting up the acedb client/server system can be simple if you just wish to test out the client/server. Setting up a database and client/server system is more complicated for a "production" system and requires setting file permissions for the server, client and database files. In addition the server configuration and password files must also be set up. Once this is done however the server system will allow access to users from other machines whether those machines are fairly local or perhaps even on the other side of the world.

You should now read the more extensive notes in http://www.acedb.org/Development/wdoc/SOCKET_aceserver.html to make sure you understand all the configuration options for the server. You should also read this documentation for descriptions of the admin commands that will enable you to add/change/delete users for a database, shutdown the database and do other administration.

In the Future

The conversion of the client/server to use sockets (rather than RPC) has opened up the server for interfacing from other programming languages/environments, in particular:

perl
see http://stein.cshl.org/AcePerl/
java
to be announced soon at http://www.acedb.org
python
to be announced soon at http://www.acedb.org

The move to sockets has also made it possible to port the client/server to MS-Windows (http://www.acedb.org/downloads/). In the future a Macintosh port of the server may also be possible.