Introduction to UCam

UCam is a software camera control and data acquisition system designed for astronomical applications using the San Diego State University (SDSU) camera hardware. SDSU hardware can read out a variety of CCDs and IR arrays, and allows multiple channels per controller, multiple controller synchronization and DSP (software) based sequencing and readout control. Connection to host computers is normally via fibre optic cards (50MHz and 250MHz versions), VME, PCI and PCI mezzanine boards are available.

Philosophy

When designing UCam there were several goals in mind, conclusions that were based on extensive use of earlier ,VxWorks based SDSU systems. The main goals we hoped to achieve were;

Simple, effecient real-time layer::

We wanted the real time layer to be as simple as possible. Previous systems had used VxWorks throughout and bug fixes had been applied at all levels. Real-time layers are difficult to code and test, and we felt that by minimising the compelxity and the RT layers knowledge of the system would improve the system.

Limited infrastructure requirements::

We wanted to make sure that the system only relied on the core software that the system needed. Where possible we wanted to package all this infrastructure together. (Prior systems had relied to heavily on huge environments, so that the user had to install, for example, EPICS in order to get the Makefile system to work).

Services

Other tools

Building and Installing

The vital parts

SDSU controllers

Real time layer

Commands to/from the SDSU systems

Commands sent between the Real-time layer (generally RTAI on Linux) and the SDSU system (the PCI card and the controller itself).

PCI commands (commands to/from the PCI board)

RDM	READ memory
WRM	WRITE memory
GOA	Application GO
STP	Application STOP
RST	PCI board software RESET
CON	Data packet to CONTROLLER
HST	Data packet to HOST
RCO	Timing board hardware RESET
BIG	BIG endian byte swapping	(1)
WEE	LITTLE endian byte swapping	(1)

*Notes* (1) : Only available when SDSU_BYTE_SWAP has been defined in the source code. This is not used for RTAI on Linux, it is available should the system be ported to other hardware.

Timing board commands (commands to/from the SDSU timing board)

RM	? Read memory ?
WM	? Write memory ?
GO	? Go ?
SP	? Stop ?
RS	? Reset ?

Replies

Replies all come from the PCI board, so they have to be examined to see if they came from the SDSU originally.

*Major*	*Minor*	*Type*	*Description*
REP	n.a.	Reply	Reply from the PCI card
NFY	*	Notify	Reply from the SDSU system
NFY	DA	Data	Reply from SDSU with data
NFY	RP	Acknowledgement	? Reply from SDSU to a sent command ?

User-space layer

The user space layer consists of two major servers (camera and filesave) and one ancillary (the data handler or demux). The camera and filesave servers are written in C++ and have two sets of interfaces. To the outside world (the user) they present an http interface that transfers commands as XML documents. To the real time layer they interface through a combination of FIFOs (for most controls) and shared memory (mainly for bulk data).

The camera server

The filesave server

Data handling interface

UCAM File class. FIFOs.

API's and Interfaces

High Level Control interface

The high level interface allows easy access from most programming languages, and even allows simple debug to be performed via a web browser. It is easy enough to develop your own API to this interface, but a simple high level API is also available which will suffice for most applications.

The control interface uses http: v1.0 connections. Simple commands are issued using a GET method where any arguments are represented in CGI form and the return value is transfered as the reply. More complex commands (generally configuration methods) are issued using a POST method. The tables below outline the available commands on each server;

Camera commands

*URL*	*Method*	*CGI args*	*Body data*	*Note*
config	POST	XML configuration
config	GET|filename
debug	GET	query-string
exec	GET
monitor	GET
status	GET
ssetconfig	GET
sunsetconfig	GET
list	GET
get	GET

Filesave methods

*URL*	*Method*	*CGI args*	*Body data*	*Note*
config	POST
config	GET
debug	GET
files	GET	unimplemented at this time.
fstatus	GET
status	GET
start	GET
stop	GET
setconfig	GET
unsetconfig

GET config?filename

Load the filename given as an XML configuration file, and return the status after parsing the file.

POST config

Parse the body of the http: POST as an XML configuration file, and return the status after parsing the file.

GET debug?query-string

Return a debug message for the given query-string. Query strings, and the information they return, are;

query-string	return
instrument	contents of the instrument structure
obs	contents of the observatory structure
app	(contents of) executable application loaded on the controller
dat	application data parameters of executable application
sdsu	SDSU context structure

GET files

Unimplemented at this time.

_Note_:the default string if none is siupplied is "instrument"

Simple API

As mentioned before, a simple API is available which links against the CURL library (for http) and libxml2 (for XML processing). The library is not yet under autoconf management, so you will need to edit the Makefile to compile it, but the code is not complicated and would be a good starting point should you wish to roll your own interface.

See source:UCamSimpleIF/trunk for the latest version.

See also

• CURL ​http://xmlsoft.org/
• libxml2 ​http://curl.haxx.se/

Configuration

XML Files

Introduction: please read this

In order to simplify writing this manual a convention has been adopted to represent XML hierachies, where tag elemnts are spearated by dots and attributes are differntiated by square brackets. So, where a piece of text says application_data.header.headerwords=12, you should assume it means the following XML fragment;

   <application_data>
     <header>
       <headerwords> 12 </headerwords>
     </header>
   </application_data>

Similarly, application_data.header.camera_status[type]=uint means;

   <application_data>
     <header>
       <camera_status type="uint">  </camera_status>
     </header>
   </application_data>

XML usage in Ultracam

The XML files in Ultracam are used to store (almost) all information about the system, from the lowest level DSP executable code to user comments about the data sets. The standard set up of Ultracam includes three levels of files;

Low level files::

these are generated from the DSP object files (.lod files) using the coff2xml tool. The file contains a root tag of type coff and subtags which decalre memory regions in the DSP (parameter_filter tags), dsp executable code (dsp_data tags) and locations of parameters (parameter tags). The parameter tags are generated directly from named variables in the DSP assembler code, in this way any named variable can be set or queried by the system.

Intermediate files::

these are hand written, and are used to apply validation to variables using condition_check tags and to describe the data format using an application_data section. There are also ancillary files which describe the SDSU hardware, telescope details etc.

Top level files::

these are the files that get sent to the system by an interface. They contain set_parameter tags (in a configure_XXX section) and (potentially) a user section

It should be noted that this three layer structure is entirely arbitrary - since the XML files reference each other (see XML Message Execution below) Ultracam could use 1 or more files. Three levels just seems to be optimal from a human point of view.

XML Message Execution

The following details the conceptual order of processing when an XML message is sent to one of the servers (camera or filesave).

TBC

Setting parameters

Parameters can be set using a <set_parameter> tag in the <configure_camera&gt section of an XML document. The syntax for the tag is <set_parameter ref="NAME" value="VALUE"/> where name is replaced by a valid parameter name and VALUE can be replaced by any valid value (which depends on the type of the parameter).

Condition checks

The XML format allows the use of condition checks to verify the Expressions are placed in the <executable_application> section and have the format;

  <condition_check when="TIME" fatal="FATAL" >
       <description>A description of this check</description>

       <expression expect="EXPECT"> EXPRESSION </expression>

       <message>A message to return on error</message>
    </condition_check>

where the following repacements are made;

• TIME can be replaced by one of the values;
  • start - check these conditions after downloading the application.
  • pre - check these conditions before executing the application.
  • post - check these conditions after executing the application.
  • reset - check these conditions after executing a reset.
• FATAL can be replaced by a boolean value (the parser supports T,Y,1 for
• true and F,N,0 for false). A true value will cause failure of this check to cause a halt in processing and return of an

error, a false value will return an error but

continue processing

• EXPECT can be replaced by a boolean value. It represents the value of the
• expression expected on success (which allows you to use either positive or negative logic in the expressions).
• EXPRESSION is a valid expression (see the next subsection)

Expression parser

The expression parser understands the following operators;

*Operator*	*XML entity*	*Meaning*
+	+	Addition
-	-	Subtraction
*	*	Multiplication
/	/	Division
%	%	Modulo
<	&amp;lt;	Less-than
>	&amp;gt;	Greater-than
		Equal-to
#	|#|Not-equal-to
&brvbar;	&brvbar;	Logical (bitwise)OR
&	&	Logical (bitwise)AND

Expressions can be built up using these operators, constant numeric values and the names of parameters. One complication is that, unless you are using an editor capabale of handling 16 bit characters (and it's very unlikely that you are), you will need to use the ISO escape sequences to enter the less-than and gretaer-than symbols. These are < and > respectively, but remove the spaces before you use them!

Example

Consider the following example;

    <condition_check when="pre" fatal="Y" >
       <description>Check for sensible number of exposures</description>
       <expression
expect="T">(NUM_EXPS&gt;0)&amp;(NUM_EXPS&lt;4096)</expression>
       <message>Invalid number of exposures.</message>
    </condition_check>

1 In the first tag, _when="pre"_ means the condition gets checked before

executing (but after downloading) the application. _fatal="Y"_

means that falure of the condition check will halt processing and return

an error.

2 In the second tag is a description of the check 3 In the third tag, _expect="T"_ means we expect this condition to evaluate

to true (otherwise it is an error). The expression itself,

_(NUM_EXPS>0)&(NUM_EXPS<4096)_, says that we expect the value

of the parameter NUM_EXP to be greater than 0 and less

than 4096.

4 The fourth tag gives the message to return if the check fails.

Data description

Data that is produced by UCam is described by data inside the

application_data xml tag. The format and contents of this section are;

• *nframes* (required) An integer or an expression that evaluates to an
• integer, specifying the number of frames. -1 is taken to mean continuous
• readout
• *header* (see below) A section containing;
  • *headerwords* An integer or an expression that evaluates to an
  • integer, specifying the size of the header in WORDS
  • *camera_status* A section containing a description of status bits
  • *header_parameter* Any number of declared named parameters, extracted
  • from header words, that will appear in the output XML
• *data* (required) A section containing;
  • *npixels* An integer or an expression that evaluates to an integer,
  • specifying the number of pixels in each output data packet
  • *ncolumns* An integer or an expression that evaluates to an integer,
  • specifying the number of columns in each data frame
  • *nrows* An integer or an expression that evaluates to an integer,
  • specifying the number of rows in each data frame
  • *window* Any number of named windows (see below)
  • *channel* Any number of named channels (see below)

two sets of XML tags, channel tags and window tags. A channel tag

describes in what order the data is transmitted from the controller, and a

window tag describes how the data should be placed

in the output.

Description of data sizes etc required.

*Note* The order of evaluation of the data packet size has been changed, it now is;

• If the data is fullframe (i.e. application_data.data[fullframe] == "yes")
• then
  • If there is a value for npixels then
    • the size of the packet is (npixels + headerwords)*wordsize
  • else
    • the size of the packet is ((ncolumns*nrows) + headerwords)*wordsize
• else
  • the size of the packet is (npixels + headerwords)*wordsize

header tag

Description of header, status bits and header parameters required.

Window tag

A window tag has the following form;

<window id="Q4W1" name="quad4win1" join="win4_1" xleft="0" ybottom="896"
xsize="1024" ysize="128" >

where the attributes have the following meanings;

id	Any sensible ID tag for the window, e.g. "Q4W1" is an ID tag for Quadrant4 Window 1.
name	Any sensible name for the window, e.g. "quad4win1" is a shortened namefor Quadrant 4 Window 1.
join	This attribute is a unique word which is used for matching the window with the channel item.
xleft	The cartesion co-ordinate for the first pixel on the X-axis of the window in the final frame.
ybottom	The cartesian co-ordinate for the first pixel on the Y-axis of the window in the final frame.
xsize	The number of pixels in the X-axis for the window size.
ysize	The number of pixels in the Y-axis for the window size.

Channel tag

A channel tag has the following form;

<channel id="31" name="qaud4chan7" chip="Hawaii2" join="win4_7" index="col"
stepcol="-1" steprow="+1" offset="30"/>

where the attributes have the following meanings;

id	Any sensible ID tag for the channel
name	Any sensible name for the window
chip	The chip that this channel is produced by
join	This attribute is a unique word which is used for matching the window with the channel item.
index	The major indexing order of the bytes, row or column
stepcol	The index increment in the column pixels
steprow	The index increment in the row pixels
offset	The increment step between consequtive pixels

Some simple examples are probably the easiest way of understanding the format. All these examples assume a single 1024x1024 sized detector called chip1;

1) There is one output channel, but all the pixels are red from the bottom right of the device

rather than the bottom left so one axis needs to be reversed;

    <window id="W1" name="window1" join="number1" xleft="0" ybottom="0"
xsize="1024" ysize="1024" >
    <channel id="C1" name="channel1" chip="chip1" join="number1" index="col"
stepcol="-1" steprow="+1" offset="0"/>

2) There are two readouts, one bottom left and one bottom right so the chip is split vertically down the center. The pixels from

left and right channels are multiplexed in the (single) output stream.

    <window id="W1" name="window1" join="number1" xleft="0" ybottom="0"
xsize="512" ysize="1024" >
    <window id="W2" name="window2" join="number2" xleft="512" ybottom="0"
xsize="512" ysize="1024" >
    <channel id="C1" name="channel1" chip="chip1" join="number1" index="col"
stepcol="1" steprow="+1" offset="1"/>
    <channel id="C2" name="channel2" chip="chip1" join="number2" index="col"
stepcol="-1" steprow="+1" offset="1"/>

3) There are two outputs as before, but now one is bottom left and is column first and the other is top right and is row

first (i.e. the left half readout scheme has been rotated by 90 degrees)

    <window id="W1" name="window1" join="number1" xleft="0" ybottom="0"
xsize="512" ysize="1024" >
    <window id="W2" name="window2" join="number2" xleft="512" ybottom="0"
xsize="512" ysize="1024" >
    <channel id="C1" name="channel1" chip="chip1" join="number1" index="col"
stepcol="1" steprow="+1" offset="1"/>
    <channel id="C2" name="channel2" chip="chip1" join="number2" index="row"
stepcol="-1" steprow="+1" offset="1"/>

If you want to see an example of 32 channels in four quadrants (8 channels per quadrant), with each quadrant rotated at 90 degrees to the first, take a look at the Hawaii II code in the Ultracam source!