[[PageOutline]] = Introduction to UCam = '''UCam''' is a software camera control and data acquisition system designed for astronomical applications using the San Diego State University (SDSU) camera hardware. SDSU hardware can read out a variety of CCDs and IR arrays, and allows multiple channels per controller, multiple controller synchronization and DSP (software) based sequencing and readout control. Connection to host computers is normally via fibre optic cards (50MHz and 250MHz versions), VME, PCI and PCI mezzanine boards are available. == Philosophy == When designing '''UCam''' there were several goals in mind, conclusions that were based on extensive use of earlier ,!VxWorks based SDSU systems. The main goals we hoped to achieve were; Simple, effecient real-time layer:: We wanted the real time layer to be as simple as possible. Previous systems had used !VxWorks throughout and bug fixes had been applied at all levels. Real-time layers are difficult to code and test, and we felt that by minimising the compelxity and the RT layers knowledge of the system would improve the system. Limited infrastructure requirements:: We wanted to make sure that the system only relied on the core software that the system needed. Where possible we wanted to package all this infrastructure together. (Prior systems had relied to heavily on huge environments, so that the user had to install, for example, EPICS in order to get the Makefile system to work). = Services = == Other tools == == Building and Installing == == The vital parts == == SDSU controllers == == Real time layer == === Commands to/from the SDSU systems === Commands sent between the Real-time layer (generally RTAI on Linux) and the SDSU system (the PCI card and the controller itself). ==== PCI commands (commands to/from the PCI board) ==== || RDM ||READ memory || || || WRM ||WRITE memory || || || GOA ||Application GO || || || STP ||Application STOP || || || RST ||PCI board software RESET || || || CON ||Data packet to CONTROLLER || || || HST ||Data packet to HOST || || || RCO ||Timing board hardware RESET || || || BIG ||BIG endian byte swapping || (1) || || WEE ||LITTLE endian byte swapping || (1) || *Notes* (1) : Only available when SDSU_BYTE_SWAP has been defined in the source code. This is not used for RTAI on Linux, it is available should the system be ported to other hardware. ==== Timing board commands (commands to/from the SDSU timing board) ==== || RM || ? Read memory ? || || WM || ? Write memory ? || || GO || ? Go ? || || SP || ? Stop ? || || RS || ? Reset ? || ==== Replies ==== Replies all come '''from''' the PCI board, so they have to be examined to see if they came from the SDSU originally. || *Major* || *Minor* || *Type* || *Description* || || REP || n.a. || Reply || Reply from the PCI card || || NFY || * || Notify || Reply from the SDSU system || || NFY || DA || Data || Reply from SDSU with data || || NFY || RP || Acknowledgement || ? Reply from SDSU to a sent command ? || == User-space layer == The user space layer consists of two major servers (camera and filesave) and one ancillary (the data handler or demux). The camera and filesave servers are written in C++ and have two sets of interfaces. To the outside world (the user) they present an http interface that transfers commands as XML documents. To the real time layer they interface through a combination of FIFOs (for most controls) and shared memory (mainly for bulk data). === The camera server === === The filesave server === === Data handling interface === UCAM File class. FIFOs. == API's and Interfaces == == High Level Control interface == The high level interface allows easy access from most programming languages, and even allows simple debug to be performed via a web browser. It is easy enough to develop your own API to this interface, but a simple high level API is also available which will suffice for most applications. The control interface uses http: v1.0 connections. Simple commands are issued using a GET method where any arguments are represented in CGI form and the return value is transfered as the reply. More complex commands (generally configuration methods) are issued using a POST method. The tables below outline the available commands on each server; === Camera commands === ||*URL*||*Method*||*CGI args*||*Body data*||*Note*|| ||config||POST||XML configuration|||| ||config||GET|filename|||||| ||debug||GET||query-string|||||| ||exec||GET|||||||| ||monitor||GET|||||||| ||status||GET|||||||| ||ssetconfig||GET|||||||| ||sunsetconfig||GET|||||||| ||list||GET|||||||| ||get||GET|||||||| === Filesave methods === ||*URL*||*Method*||*CGI args*||*Body data*||*Note*|| ||config||POST|||||||| ||config||GET|||||||| ||debug||GET|||||||| ||files||GET||||unimplemented at this time.|| ||fstatus||GET|||||||| ||status||GET|||||||| ||start||GET|||||||| ||stop||GET|||||||| ||setconfig||GET|||||||| ||unsetconfig|||||||| ==== GET config?filename ==== Load the filename given as an XML configuration file, and return the status after parsing the file. ==== POST config ==== Parse the body of the http: POST as an XML configuration file, and return the status after parsing the file. ==== GET debug?query-string ==== Return a debug message for the given query-string. Query strings, and the information they return, are; ||query-string||return|| ||instrument||contents of the instrument structure|| ||obs||contents of the observatory structure|| ||app||(contents of) executable application loaded on the controller|| ||dat||application data parameters of executable application|| ||sdsu||SDSU context structure|| ==== GET files ==== Unimplemented at this time. _Note_:the default string if none is siupplied is "instrument" == Simple API == As mentioned before, a simple API is available which links against the CURL library (for http) and libxml2 (for XML processing). The library is not yet under autoconf management, so you will need to edit the Makefile to compile it, but the code is not complicated and would be a good starting point should you wish to roll your own interface. See source:UCamSimpleIF/trunk for the latest version. === See also === * CURL http://xmlsoft.org/ * libxml2 http://curl.haxx.se/ == Configuration == == XML Files == === Introduction: please read this === In order to simplify writing this manual a convention has been adopted to represent XML hierachies, where tag elemnts are spearated by dots and attributes are differntiated by square brackets. So, where a piece of text says __application_data.header.headerwords=12__, you should assume it means the following XML fragment; {{{
12
}}} Similarly, __application_data.header.camera_status[type]=uint__ means; {{{
}}} === XML usage in Ultracam === The XML files in Ultracam are used to store (almost) all information about the system, from the lowest level DSP executable code to user comments about the data sets. The standard set up of Ultracam includes three levels of files; Low level files:: these are generated from the DSP object files (.lod files) using the coff2xml tool. The file contains a root tag of type '''coff''' and subtags which decalre memory regions in the DSP '''(parameter_filter''' tags), dsp executable code '''(dsp_data''' tags) and locations of parameters '''(parameter''' tags). The parameter tags are generated directly from named variables in the DSP assembler code, in this way any named variable can be set or queried by the system. Intermediate files:: these are hand written, and are used to apply validation to variables using '''condition_check''' tags and to describe the data format using an '''application_data''' section. There are also ancillary files which describe the SDSU hardware, telescope details etc. Top level files:: these are the files that get sent to the system by an interface. They contain '''set_parameter''' tags (in a '''configure_XXX''' section) and (potentially) a '''user''' section It should be noted that this three layer structure is entirely arbitrary - since the XML files reference each other (see XML Message Execution below) Ultracam could use 1 or more files. Three levels just seems to be optimal from a human point of view. == XML Message Execution == The following details the conceptual order of processing when an XML message is sent to one of the servers (camera or filesave). '''TBC''' === Setting parameters === Parameters can be set using a <set_parameter> tag in the <configure_camera> section of an XML document. The syntax for the tag is <set_parameter ref="NAME" value="VALUE"/> where name is replaced by a valid parameter name and VALUE can be replaced by any valid value (which depends on the type of the parameter). === Condition checks === The XML format allows the use of condition checks to verify the Expressions are placed in the <executable_application> section and have the format; {{{ A description of this check EXPRESSION A message to return on error }}} where the following repacements are made; * TIME can be replaced by one of the values; * start - check these conditions after downloading the application. * pre - check these conditions before executing the application. * post - check these conditions after executing the application. * reset - check these conditions after executing a reset. * FATAL can be replaced by a boolean value (the parser supports T,Y,1 for * true and F,N,0 for false). A true value will cause failure of this check to cause a halt in processing and return of an error, a false value will return an error but continue processing * EXPECT can be replaced by a boolean value. It represents the value of the * expression expected on success (which allows you to use either positive or negative logic in the expressions). * EXPRESSION is a valid expression (see the next subsection) === Expression parser === The expression parser understands the following operators; ||*Operator*||*XML entity*||*Meaning*|| ||+||+||Addition|| ||-||-||Subtraction|| ||*||*||Multiplication|| ||/||/||Division|| ||%||%||Modulo|| ||<||&lt;||Less-than|| ||>||&gt;||Greater-than|| ||=||=||Equal-to|| ||#|||#|Not-equal-to|| ||¦||¦||Logical (bitwise)OR|| ||&||&||Logical (bitwise)AND|| Expressions can be built up using these operators, constant numeric values and the names of parameters. One complication is that, unless you are using an editor capabale of handling 16 bit characters (and it's very unlikely that you are), you will need to use the ISO escape sequences to enter the less-than and gretaer-than symbols. These are __&lt;__ and __&gt;__ respectively, but remove the spaces before you use them! ==== Example ==== Consider the following example; {{{ Check for sensible number of exposures (NUM_EXPS>0)&(NUM_EXPS<4096) Invalid number of exposures. }}} 1 In the first tag, _when="pre"_ means the condition gets checked before executing (but after downloading) the application. _fatal="Y"_ means that falure of the condition check will halt processing and return an error. 2 In the second tag is a description of the check 3 In the third tag, _expect="T"_ means we expect this condition to evaluate to true (otherwise it is an error). The expression itself, _(NUM_EXPS>0)&(NUM_EXPS<4096)_, says that we expect the value of the parameter NUM_EXP to be greater than 0 and less than 4096. 4 The fourth tag gives the message to return if the check fails. === Data description === Data that is produced by UCam is described by data inside the __application_data__ xml tag. The format and contents of this section are; * *nframes* (required) An integer or an expression that evaluates to an * integer, specifying the number of frames. -1 is taken to mean continuous * readout * *header* (see below) A section containing; * *headerwords* An integer or an expression that evaluates to an * integer, specifying the size of the header in WORDS * *camera_status* A section containing a description of status bits * *header_parameter* Any number of declared named parameters, extracted * from header words, that will appear in the output XML * *data* (required) A section containing; * *npixels* An integer or an expression that evaluates to an integer, * specifying the number of pixels in each output data packet * *ncolumns* An integer or an expression that evaluates to an integer, * specifying the number of columns in each data frame * *nrows* An integer or an expression that evaluates to an integer, * specifying the number of rows in each data frame * *window* Any number of named windows (see below) * *channel* Any number of named channels (see below) two sets of XML tags, __channel__ tags and __window__ tags. A channel tag describes in what order the data is transmitted from the controller, and a window tag describes how the data should be placed in the output. Description of data sizes etc required. *Note* The order of evaluation of the data packet size has been changed, it now is; * If the data is fullframe (i.e. application_data.data[fullframe] == "yes") * then * If there is a value for npixels then * the size of the packet is (npixels + headerwords)*wordsize * else * the size of the packet is ((ncolumns*nrows) + headerwords)*wordsize * else * the size of the packet is (npixels + headerwords)*wordsize ==== header tag ==== Description of header, status bits and header parameters required. ==== Window tag ==== A window tag has the following form; {{{ }}} where the attributes have the following meanings; ||id||Any sensible ID tag for the window, e.g. "Q4W1" is an ID tag for Quadrant4 Window 1. || ||name||Any sensible name for the window, e.g. "quad4win1" is a shortened namefor Quadrant 4 Window 1. || ||join||This attribute is a unique word which is used for matching the window with the channel item. || ||xleft||The cartesion co-ordinate for the first pixel on the X-axis of the window in the final frame. || ||ybottom||The cartesian co-ordinate for the first pixel on the Y-axis of the window in the final frame. || ||xsize||The number of pixels in the X-axis for the window size. || ||ysize||The number of pixels in the Y-axis for the window size. || ==== Channel tag ==== A channel tag has the following form; {{{ }}} where the attributes have the following meanings; ||id||Any sensible ID tag for the channel|| ||name|| Any sensible name for the window|| ||chip|| The chip that this channel is produced by|| ||join|| This attribute is a unique word which is used for matching the window with the channel item.|| ||index|| The major indexing order of the bytes, row or column|| ||stepcol|| The index increment in the column pixels|| ||steprow|| The index increment in the row pixels|| ||offset|| The increment step between consequtive pixels || Some simple examples are probably the easiest way of understanding the format. All these examples assume a single 1024x1024 sized detector called chip1; 1) There is one output channel, but all the pixels are red from the bottom right of the device rather than the bottom left so one axis needs to be reversed; {{{ }}} 2) There are two readouts, one bottom left and one bottom right so the chip is split vertically down the center. The pixels from left and right channels are multiplexed in the (single) output stream. {{{ }}} 3) There are two outputs as before, but now one is bottom left and is column first and the other is top right and is row first (i.e. the left half readout scheme has been rotated by 90 degrees) {{{ }}} If you want to see an example of 32 channels in four quadrants (8 channels per quadrant), with each quadrant rotated at 90 degrees to the first, take a look at the Hawaii II code in the Ultracam source!