"find model"
"list"
etc.
can be passed to "interpreter" code which will then call the low level
database routines required to fulfill those commands. It is at this level that
the split between client and server is made. The client reads commands
typed in by the user
(or perhaps read from a file containing a collection of commands
that the user has supplied), and then passes these commands across the
network to the server. The server contains the "interpreter" code which
scans the commands and makes calls to the low level database routines
to access the actual database.
The server then constructs a reply as plain text which it sends back to the
client, the client displays this reply to the user. In this way there is
no difference really between tace and aceclient, both are command line
based, it's just that aceclient accesses the database via a server
across the network.
These interfaces will not be demonstrated here, the former because it is experimental, the latter because it is quite complex to set up and beyond the scope of this course.
If you are familiar with Unix/computers then skip this paragraph, otherwise..... We tend to think of a program as the single file on the disk that contains that program. It is more correct however to think of "instances" of a running program: many copies of the same program can be running at the same time. This is important when thinking about the server because each time a server is started up it can only access one database. Hence to allow access to many databases on a single machine, there may be many instances of the server program running at the same time, each accessing a different database.
Now, which instance of the server do you contact to access a particular database ? To understand this we need to know a bit about port numbers.
Computers make sure that the many different types of communications they receive over the network go to the correct programs by associating each program with a unique "port" number, e.g. if you look in the file /etc/services you can see a list of port numbers/programs, some of which are essential operating system services. For now all you need to know is two things:
You can probably already see that the port number is like a "key" to the database. When the client contacts a server on a particular port number, the server on the other end of that port will be accessing a particular database. To access a different database we could start up another instance of a server running on a different port and using a different database.
So when you start up the client you do not say which database you want to access, instead you specify the machine you want to contact and which port on the machine you want to connect to.
(NOTE that the client and server can be on the same or different machines, the commands are the same.)
(In this and other examples "YOURDB" stands for a whatever directory you may have set up your database in, e.g. /home/johnsmith/mydatabase, so that if you see "YOURDB/wspec" it means /home/johnsmith/mydatabase/wspec)
Let's say we are working on the machine testbed and start up the server like this:
> aceserver YOURDB 2000
then the client can be started up like this:
> aceclient testbed -port 2000
The client will now be able to access YOURDB via the server on testbed on port 2000. Note how the client needs to be told the machine and port number and then the database that will be accessed is implicit from this because of how the server was started up.
There are several things you need to do before you can try out the client/server code:
For this initial exercise we will set up the client/server as simply as possible, glossing over some of the details (see Appendix A for a full description).
We'll assume that step 1) has been done and that the database has been set up by you in some directory of your choice, e.g. YOURDB
At this stage we will use the simplest option and make the database readable by anybody so that we can get on and try out the server/client. To do this you need to edit the file YOURDB/wspec/server.wrm and edit the line that begins READ_ACCESS_DIRECTORY so that it looks like this:
READ_ACCESS_DIRECTORY PUBLIC
This tells the server to let anyone have read access to the database.
The simplest way to start the server is simply by starting it at a terminal, this is what we will do. Make sure you have at least two xterms running on your X Windows screen. Decide what port number you want to use (must be greater than 1023 and not in /etc/services) and then start the server by typing:
> aceserver /your/database/directory your_port_number
If the server has started correctly then the xterm prompt will not be redisplayed, the xterm will "hang" until the server program exits.
You now need to start the client in a separate xterm window. Since you have started the server on your own machine you now need to know what the name of your machine is. You can find this out by typing the command "hostname":
> hostname
this should return the name by which your machine is known on the network.
Now start the client by typing:
> aceclient your_machine_name -port your_port_number
If you are successful then you should see output like this:
acedb@your_machine_name>
you can now type commands in at the "acedb" prompt to explore the database.
tace and aceclient share the same command interface which enables you to to interact with the database, do the following to get an idea of the scope of the interface:
acedb@your_machine_name> ?
- will give you a list of all possible commands
acedb@your_machine_name> help
- will give you a brief overview of the sets of commands and what they do
acedb@your_machine_name> help topic
- will give you more detailed help on a particular topic/command
Try out a few commands:
acedb@your_machine_name> classes
- this gives a list of all the visible class names and how many objects they contain
To see the model or schema for a class try the "model" command:
acedb@your_machine_name> model oligo
- will show the model for the oligo class, you should see something like:
?Oligo Sequence UNIQUE Text // verbatim sequence - useful
In_sequence ?Sequence XREF Oligo
STS STS1 UNIQUE ?STS XREF Oligo1
STS2 UNIQUE ?STS XREF Oligo2
When you want to stop the client simply type:
acedb@your_machine_name> quit
You could try getting access from another machine by using an xterm on someone elses machine and once again issuing the command
> aceclient your_machine_name -port your_port_number
Note how the same command gets you access to the server from a completely different machine. All that is needed are the machine name and port number. Provided your network security system allowed it, you could even use your server from a machine in a different country although you would need to use the long form of the machine name, e.g. your_machine_name.hgmp.ac.uk or whatever.
To stop the server, go to the xterm running the server and press and hold down the 'Ctrl' key and press the 'c' key. This will send a special signal to the server program that will cause it to exit (later you will learn of a better way to stop the server).
To recap, you have learnt the basic set up of the database and server/client programs, how to start the programs, basic use of the client and how to stop the programs. The following sections will discuss in more detail how to control access to a database, how to shut it down properly, how to monitor the server, and how to tell the operating system to start up the server automatically every time a client connects to the port for your database.
So far you have only given read access to clients signing on. Now we will go through the basic steps to get write access (once again glossing over some of the details in the interests of making things simple to start off with).
The server controls access in the following way:
When a client contacts the server, the server writes a file containing a special number into a directory that is on the shared file system common to both the servers and the clients machines. (Since NFS comes as standard with most Unix systems this should not be a problem.) If the userid of the client has the correct Unix file permissions it will be able to read this file and get the special number contained in the file. The client can then return this number to the server which acts like a key and will tell the server that it is OK to give the client access to the database. An example will show how this works:
> mkdir your_directory_name
> chmod ugo+rwx your_directory_name
WRITE_ACCESS_DIRECTORY your_directory_name
(Note that you must give the full path name of the directory. )
When the server is contacted by a client it will now write the file containing the magic number in this directory. Your client will then be able to read it and send the number back to the server to get write access. Now we need to try this out.
Start the server as before:
> aceserver /your/database/directory your_port_number
Now start the client:
> aceclient your_machine_name -port your_port_number
To find out if you have write access, type in the "save" command:
acedb@your_machine_name> save
If you have write access you will see a message like this:
// Response: 24 bytes.
// 0 Active Objects
If not, you will see a message explaining that you do not have write access:
// Reponse: 64 bytes.
// Sorry, you do not have Write Access.
// 0 Active Objects
If you did not get write access then check through each step carefully and try again.
Although we have used this mechanism to provide write access we can also use the same mechanism to control read access by creating a new directory and using it with the READ_ACCESS_DIRECTORY keyword (in wspec/server.wrm) instead of the special "PUBLIC" keyword we used in the previous example. The server will check first for write access and then for read access.
You should also note that access can be very strictly controlled (in contrast to what we have done) so that only certain groups of users can have read or write access. Appendix A to this document explains how to do this using Unix group and file permissions. It's enough for now to note that the following combinations are possible:
READ ACCESS
no one group anyone
no one not allowed OK OK
WRITE group OK OK OK
ACCESS
anyone OK OK not allowed
The only two combinations disallowed are that no one has read/write access or that everyone has read/write access, the former renders the database inaccessible, the latter is just not a good idea !!
In a previous example we used Cntl-C to send a signal to the server which forced it to exit. This should only be used as a last resort if you are running the server on an important database. It is much better to use the "shutdown" command which will cause the server to refuse any new client connections and close down the database in an orderly way once the last client has quit. Only users with write access permission are allowed to do this, you could try the command out now that you know how to get write access.
There are two files used by the server to log errors/information:
YOURDB/database/log.wrm
YOURDB/server.log
log.wrm is used by all acedb programs that access the database to record information and errors. If your program crashes then the cause of the crash will be recorded in this file unless the crash was so bad that the program was forced to exit immediately. xace, tace and the aceserver all log errors in this file.
server.log is used by the server to record useful information that may help with debugging or show how many clients have logged on and so on.
You should look in these files if you have problems such as the server not starting, not responding normally or crashing.
It is also useful to use the Unix ps command to look at what the server process is doing, assuming the server program is still called aceserver:
> ps -l
will display something like this:
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
8001 S 9767 17388 17337 1.2 44 0 0 7.3M event ttyp8 0:00.35 aceserver
< lots more process entries >
The most useful fields are the SZ and WCHAN fields:
SZ will tell you how much memory the server is using, if you suspect that the server is running away and eating up more and more memory it will be obvious here (the aceserver is 7.3MB in the above example). WCHAN gives you an idea of what the server process is doing, in the above example it is waiting for an "event", i.e. the server is effectively asleep and waiting for a client to connect.
You could try out this command before, during and after a client connection.
"top" is another useful command if your system supports it. It will show you a dynamic display of the top users of CPU on your system. If you suspect the server is looping out of control you can use this command to check this. In the below example you can see that the aceserver is not currently active.
> top
will display something like this:
-----------------------------------------------------------------------
load averages: 0.19, 0.08, 0.07
109 processes: 1 running, 35 sleeping, 73 idle
CPU states: 7.5% user, 0.0% nice, 8.2% system, 84.2% idle
Memory: Real: 298M/494M act/tot Virtual: 38M/852M use/tot Free: 33M
PID USERNAME PRI NICE SIZE RES STATE TIME CPU COMMAND
16721 lmc 44 0 33M 19M sleep 0:52 3.10% real-netscape
640 root 42 -2 16M 9928K sleep 2:47 0.40% Xdec
17389 edgrif 44 0 4576K 745K sleep 0:00 0.20% aceserver
etc. etc.
As well as the help files in YOURDB/whelp, you will find additional information about various aspects of programming and installing ACEDB in YOURDB/wdoc. (See the document RPC_aceserver.html for server details) You can also find more documentation from the acedb web site (http://www.acedb.org) which is hosted at the Sanger site and can be accessed from there as well (http://www.sanger.ac.uk).
So far we have run the server by starting it from an xterm, this is fine for testing but not satisfactory for providing a database service because we would like users to have access to the database whether we are logged on to the machine or not. For this we would like to have server startup automated and fortunately on Unix systems this can be done via an operating system service called inetd (which is short for "Internet Daemon"). inetd is a program that will start up server programs such as the aceserver when a client request arrives at the servers machine. Briefly it does this by listening on all the ports that these servers would normally use and then when a request comes in on a port it starts up the server that should service that port and lets it deal with the request. This section shows what you need to do to have inetd start up the aceserver.
**** TAKE CARE ****
Because you will either need root access on your machine or you will need to get your system support people to do this, we will not actually edit the required files. If you want to try this out you should make sure that you do this with the help of someone who knows how to safely edit operating system files, it is possible to leave your system in a state where it may not reboot if you do not alter these files correctly.
**** TAKE CARE ****
For this example we will assume the following:
our machine name is: redbox our server program is the file: /home/me/aceserver the database we want to access is: /home/mydatabase the port we are going to use is: 2001 the nickname for this server/database is: allgenes the owner of the server program is: joebloggs(you will see these bits of information highlighted in the examples below)
We will need to alter several files in the /etc directory which is where the configuration files for most Unix operating systems are kept.
The first step is to tell the operating system which port you want to use, what protocol to use and what symbolic name we want to give to our service, to do this we need to add the following line to the bottom of /etc/services:
allgenes 2001/tcp
Now we need to tell rpc which port we are using and what our symbolic name is by adding this line to the bottom of the file /etc/rpc:
allgenes 2001
Finally we need to tell inetd which program to start up when a request comes in on the port we have named
allgenes stream rpc/tcp wait joebloggs /home/me/aceserver (contd..)
aceserver /home/mydatabase 2001
Any bits that are not highlighted will be the same for any aceserver entry in these files, remember that you will need a new set of entries for each new database you want to make available via a server.
0 to 1023: are reserved for operating system use. DO NOT USE THESE. 1024 to 49151: are "registered" ports. If you setting up the server to be run by inetd you should use one of these (you will need to check in /etc/services that your number does not clash with one already there) 49152 to 65535: are for casual/ephemeral use. You should use one of these if you are just trying out the server from an xtermNow you should be able to access the server in the same way as before by typing in:
> aceclient redbox -port 2001
If you have trouble then check you all the steps above and when you are sure they are correct then refer to the trouble shooting guide in YOURDB/wdoc/RPC_aceserver.html
In acedb 4_8 a sockets based version of the server will be released, although the interface is essentially the same there are a number of differences: