This chapter covers qdsrv (the QEF server), its client programs, and its uses, and the other QEF server qhost. All the provided services are described, but the emphasis is on the use of the path database to navigate the trees. |
The QEF server qdsrv provides five services to other programs in the QEF system: Path Database, Time Server, Version Number Database and Server, Variable Database Server, and User Path Identifier Server. Descriptions of these services follow. | |||||||||
Path Database | Enterprise-wide software development encompasses a number of platforms, directories, developers, and files -- too many for most people to remember. qdsrv maintains a database of paths of project trees which can be queried by qds and its alias qd. This use of qdsrv is covered extensively below and in other chapters. | ||||||||
Time Server |
Given the common use of distributed file systems, it is not
uncommon for builds to be done on a host other than the one
that hosts the source file system.
As such it is crucial that all the machines involved
in the builds have approximately the same time.
As part of its initialization, qef will retrieve the
time of qdsrv's host.
If the retrieved time is not within 10 seconds of the qef's
host, qef will abort as in:
% qef # being run on matt #{ @matt 2000/03/01 02:59:45 qef: Time check failed 2000/03/01 02:59:59 gobo (qdsrv host) 2000/03/01 02:59:45 matt (local host) 14 difference #} E-1 matt@/m/qtree/linux2_0i/o9.1 2000/03/01 02:59:45(0) % qdmgt -t # check the time synchronization Time check failed 2000/03/01 03:06:20 gobo (qdsrv host) 2000/03/01 03:06:06 matt (local host) 14 differenceThere is a qef flag (t) that causes qef to suppress the time check, but this is inadvisable. |
||||||||
Version # Database and Server |
The program mkvernum and its link vcc are used to
create version files such as <qtree>/relinfo or the
version string in qef itself built at link time:
% qtreeinf # show Q-Tree version info $QTREE: /m/qtree/linux2_0i/9.1 Default $QTREE: /usr/qtree9.1 0: /m/qtree/linux2_0i/9.1 Real Dir: /m/qtree/linux2_0i/9.1 # following is extracted from the relinfo file Qtree(full) 9.1.6(677) Linux-2.0.36-i686(linux2_0i) 2000/03/24 dt on matt in /m/qtree/linux2_0i/o9.1 at 02:27:24 EDT, 2000/03/24 Using: Tcl/tk 7.4(12) Linux-2.0.36-i686 2000/03/09 % qef -V # get qef's version string Version: Qef (257) Qtree(9.1) 2000/02/27 07:02:20 Linux-2.0.34-i686 % wot -vp qef # wot's is somewot like SCCS's what(1) Qef (179) Qtree(9.1) 2000/02/27 07:02:20 Linux-2.0.34-i686 Libqdsrv (31) Qtree(9.1) 2000/01/02 05:49:37 Linux-2.0.34-i686 Libdtree 9.1.1(233) Qtree(9.1) 2000/02/27 07:00:02 Linux-2.0.34-i686The numbers in parentheses are unique for the module. mkvernum fetches the count for the module from the qdsrv which increments the count for the next request. qdsrv will also record the request and build information in the associated database <qtree>/mkvernum.db as in: qtree:qef 9.1(179) Linux-2.0.34-i686 gobo /p/qtree/linux2_0i/o9.1/qef dt 2000/02/27 07:02:20
|
||||||||
Variable Database Server |
The qdsrv also provides management and access to arbitrary
variables, the values of which are then available throughout the network.
Currently the only use of these variables is by qremote
-- see Chapter 10.1.
qremote retrieves variables of the form host.qtree
which stores the directory name of the Q-Tree on the specific host.
The list of such variables can be retrieved using the qdmgt
message vlist as in:
% qdmgt vlist m\*.qtree matt.qtree = /m/qtree/linux2_0i/9.1 mokey.qtree = //C/qtree8_4Variables can be deleted (using vdel), set (using vset), and retrieved (using vget). |
||||||||
User Path Identifier Server |
In Using qd in the third tutorial of
Chapter 4, an example of the use of user identifiers is given.
Using qdid s or qds I the user can
bind a path to an qdsrv identifier.
This path number can then be retrieved using qds i.
This facility allows the user to establish a pseudo home that
can be easily reached on any host on which the path is available.
For example:
% qd -pguide -o # chdir to guide project's object tree Using 59 guide 9.1 object dt gobo linux2_0i /p/guide/o /p/guide/s % qdid -s # bind current tree to my identifier Id 2: 59 guide 9.1 object dt gobo linux2_0i /p/guide/o /p/guide/s % qdmgt listids # list current path ids in use bruce: 0 path: 32 root: 1 path: -1 # -1 means no path assigned dt: 2 path: 59 % qd -i # chdir to path bound to id Using 59 guide 9.1 object dt gobo linux2_0i /p/guide/o /p/guide/s
|
||||||||
Godir Monitoring | The programs qefgui and gomonitor both monitor go and josh jobs by scanning the user's /.qtree/goes directory. This is an expensive process (particularly on platforms that have slow file systems) which would consume about 2% of the CPU if the scan was done every five seconds. Rather than scan the goes directory every five seconds, all the processes that change the goes directory or its contents send a "godir user" message to qdsrv. qdsrv preserves the time of the last "godir" message for each user. gomonitor and qefgui sends "godirget user" messages to qdsrv every five seconds and qdsrv returns the time of the last "godir user" message. The client program can then compare that time against the last time it scanned the goes directory, thereby avoiding the scan if nothing has changed. | ||||||||
Cycle Server Choice | qremote supports a Sservice flag which causes it to select the remote host to run the job from those listed by the qvrs variable _Servers_[service]. To choose from the list, qremote sends a "cycle" message to qdsrv with the list of hosts as the arguments. qdsrv maintains an internal list of hosts named as "cycle" message arguments an index of the last time it was selected. qdsrv will check the "cycle" message arguments, find the host with the lowest selected index, sets its index and returns its name to qremote. This mechanism is provided so that qremote will select a different host than the previous instances of qdsrvs until all the hosts in the list have been cycled. | ||||||||
Exit Status Communi- cation |
Unfortunately rsh has had a deficiency since
its creation: rsh does not exit with the exit status of
the argument command.
As such this creates problems for make-like processes such as
mimk which use the exit-status of its recipes to determine
if the session should be terminated.
To deal with this deficiency, qdsrv
provides the following exstat* messages:
|
Ideally, qdsrv should be started automatically on a machine accessible from every development machine -- see Appendix A.4: QDSRV Setup. The qdsrv program handles a single database that documents all trees on all projects; therefore it should be located on some machine accessible by all development platforms. To determine if qdsrv is running, issue the command: % qdmgt -v # if qdsrv is running ... Qdsrv 9.1(162) - 2000/01/19 01:08:00 - Linux-2.0.34-i686 pid=455; uid=0; host=gobo; port=6631 Database: <qtree>/data/qdsrv.db # If qdsrv is not running ... qdmgt: connect failed -- Connection refused check that qdsrv is running -- See errQdsrv1(x-qmisc) If you do see a response similar to the latter one above, try running qdsrv yourself: % qdsrv qdsrv started: errors will be logged in <qtree>/data/qdsrv.log # or qdsrv: Current host (matt) not Server host (gobo)The second message indicates that you are not on the host for which qdsrv is configured. The host name is stored in the traits variable QDSRV_HOST in <qtree>/data/traits.ext. The following command will output the current settings: % traits QDSRV_HOST QDSRV_PORT QDSRV_HOST gobo QDSRV_PORT 6631 The server qdsrv is run on some machine on the network attached to a TCP/IP port. The names of the host and the number of the port are specified by the above traits variables. An environment variable QDSRV_ID could be used to override the traits settings, but given the universal nature of the server, doing so is strictly a local temporary fix. The settings should be the same on all systems and arrangements should be made to ensure that qdsrv is always running on the named host. |
Most interaction with the server is through a few specialized commands. |
|
mkvernum | Creates release/version/time stamp files and uses qdsrv to record build records and to provide build numbers. |
qdchk | Check the qdsrv database paths for validity and consistency. |
qdid | Sets, displays, or assigns a user's path identifier. |
qdmgt | A generalized mechanism to send arbitrary qdsrv messages. qdmgt has some special purpose options: |
qds | Retrieves paths matched by arguments and default fields (used to reduce the number of matches) from the qdsrv path database. qds supports a d flag to output a directory name that may be used as an argument to cd or pushd. See qd descriptions below. |
qdupd | Add the current tree or an argument-specified path to the database -- automatically invoked by rootvrs. |
qef | The primary QEF driver. It accesses qdsrv services to check the time. |
qremote | Runs commands, possibly on a remote host. Uses qdsrv to retrieve locations of the Q-Tree on the target host so as to be able to build a $PATH independent rsh command. |
vcc | A link to mkvernum that compiles the file that it builds -- see above. |
qdsrv loads the database <qtree>/data/qdsrv.db
which consists of path records containing the following fields:
<Index> <Project> <Release> <Type> <User> <Host> <Config> <RootPath> ... as in: 13 qtree 7.1 object dt sark linux /kent/u/qtree/o /u/qtree/s |
|||||||||||||||||||||||||||||||||||||
Normal Uses |
The main purpose of qds is to provide a directory name that can be used as an argument to chdir or to pushd. The arguments to qds (or the alias qd) are used to select the path. For example: % qds -pguide -a -S # all paths for the guide project 52 guide 9.1 baseline dt gobo - /p/guide/s 59 guide 9.1 object dt gobo linux2_0i /p/guide/o /p/guide/s 63 guide 9.1 object dt matt linux2_0i /m/guide/o /m/guide/s % qds -t object # guide object trees 59 guide 9.1 object dt gobo linux2_0i /p/guide/o /p/guide/s # query automatically assumes the current project, user, # and host if doing so reduces number of paths matched % qds -o -d # output first directory only /p/guide/o/guide
If there are multiple matches, qds will apply the matches current project, current user, and current host to try to reduce the list to a single entry. These matches are equivalent to the flags p, u, and h, each with the argument ".". The d flag forces the output to be a single directory. When d is given and qds cannot reduce the list to a single directory, qds outputs the list to standard error, as in: % qds -d -o -pqtree The following multiple directories matched: 32 qtree 9.1 object dt gobo linux2_0i.f /p/qtree/linux2_0i/o9.1 /p/qtree/s9.1 48 qtree 9.1 object dt gobo linux2_0i.p /p/qtree/linux2_0i/o9.1p /p/qtree/s9.1 qds: Use the index in the first column to resolve ambiguity Most of the time, a unique match can be found with just one or two flags. And after a few uses, you start to know the index values for the paths you commonly use -- you can also specify a path directly with the index, and this may be faster. For example, to go to the path with index 32: % cd $(qds -d -32) |
||||||||||||||||||||||||||||||||||||
The qd Alias/Function |
Rather than frequently typing "cd $(qds -d ...)", we suggest you define a shell function or csh alias as follows: qd () { qdtmp=$(qds -d $*) && cd $qdtmp } # in sh, ksh or bash alias qd 'qdtmp=`qds -d \!*` && cd $qdtmp'# in csh This text is also available in the qfunc database, thus can be used to set the function or alias: % eval $(qfunc qd) # in sh, bash, or ksh % eval `qfunc -c qd` # in csh Once qd is defined, the command qd will cd to the appropriate directory in the selected path. If the selection does not result in a single path, qd will list the matching paths, including their indexes.
|
||||||||||||||||||||||||||||||||||||
Special Cases |
There are a number of special case qds flags in addition to the queries:
The i may be followed by a decimal index indicating which root in the RootPath should be selected, thus typically yielding:
Often you will know the indices of the paths you use most commonly, but if not use qds as in: % qds -p <project name> -o This command produces a list of the paths defined for the project. Then you can run: % qd -I -N where N is the index of the path of interest. This changes directory to the project root and sets the identifier. |
||||||||||||||||||||||||||||||||||||
qds Query Arguments |
qds/qd supports a variety of flags to select paths:
The above may seem formidable, however, in most instances it is sufficient to specify the ttype argument (or one of the [bmosvw] short cuts). This is because qds d will try to reduce the number of matched patterns by applying p., u., and h. queries in turn if they reduce the number of patterns matched (provided not all matches are eliminated). |
||||||||||||||||||||||||||||||||||||
qds -d argument |
qds d and qd take an optional argument that specifies the sub-directory to be appended to the root of the selected path. By default, if the selected path has the same @Project setting as the current directory, qds d will append the current displacement (i.e., @Branch) to the target root. This facilitates changing from a project tree (e.g., the object tree) to the corresponding location in another of the project's trees (e.g., baseline tree). If the corresponding directory does not exist in the target tree, sub-directories are removed until one that does exist is found. If the target tree does not have the same @Project as the current value, qds d will use the root of the target tree. If an argument is specified, it is interpreted as a sub-directory of the target directory, unless the argument begins with `/', in which case it is interpreted as a sub-directory of the target root. qds d |
||||||||||||||||||||||||||||||||||||
Some Useful qdsrv Client Commands |
|
qhost is a server that has to be run on any host that might be
a target host for qremote.
Its primary purpose is to indicate to the client that its host is up and running. It also returns the system name, pathname of the Q-Tree, and the home directory for the user on the host. So: % qhost gobo gobo: fbsd4_9i /g/qtree/fbsd4_9i/9.1 /g/dt qremote sends messages to check that hosts are available and to retrieve the Q-Tree path and the user's home directory. Relevant qhost commands are: % qhost -S # Start the server on named host % qhost -R # Keep trying to start the server when # address was busy % qhost -q host # tell host's qhost to quit % qhost -v host # retrieve version of qhost run on host |
c061.qh - 9.8 - 03/11/06 |