Reading Data

Alas, the data are stored in UDF (as of this writing). I've tried to abstract things so you don't need to know too much about UDF, though.

The simplest way to start is by using READ_IM:

    Mena> d = read_im( 'ImmStats', [2000,162,11], [2000,162,11,30] )
    [1651 records read]
    Mena> help, d, /struct
    ** Structure IMMSTATS, 29 tags, length=13360:
       BTIME           STRUCT    -> UDF_TIME Array[1]  -------\
       ETIME           STRUCT    -> UDF_TIME Array[1]  these   |
       D_QUAL          DOUBLE           2.0000000       are    |
       START_AZ        FLOAT     Array[128]         common to  |
       STOP_AZ         FLOAT     Array[128]         all UDFs   |
       SCAN_INDEX      FLOAT     Array[128] ------------------/
       START_HEIGHT    FLOAT     Array[128] ------------------\
       STOP_HEIGHT     FLOAT     Array[128]   here on, stuff   |
       START_POSITION  FLOAT     Array[128]    specific to     |
       STOP_POSITION   FLOAT     Array[128]     IMMSTATS       |
       COARSE_AZIMUTH  FLOAT     Array[128]                    |
       FINE_AZIMUTH    FLOAT     Array[128]                    |
       TOTAL_AZIMUTH   FLOAT     Array[128]                    |
       TIME_OF_FLIGHT  FLOAT     Array[128]                    |
       HEAD_ID         FLOAT     Array[128]                    |
        [ ... ]                                                :

Selecting Data

Sigh. UDF, as you know, is not one of my favorite things. Here are just two reasons why:

• Even when you specify the desired starting/ending times, you can wind up with data points outside your requested range.
• For some datasets, it only makes sense to start reading on a spin boundary, but UDF doesn't always do this.

The long and the short of it is that what READ_IM gave you is not necessarily kosher, so you need to clean it up a bit.

Let's deal with ImmStats (statistics). What I tend to do, after reading in data, is finding the unique spin identifiers:

    Mena> u = uniq(d.spin_counter[0])

This gives you a list of indices into d, not a list of spins! You can then use this list to extract a subset of d:

    Mena> d = temporary( d[u[1]:u[N_Elements(u)-2]] )

It is actually very important that you do this, since the first and last few records of d will probably comprise only part of a spin. Since they are incomplete, they should be tossed.

The temporary() is a memory-saving measure, useful when assigning to the same variable. If you were creating a new variable to hold your subselection, you must not use temporary().

You can do other stuff with u, e.g., selecting only three spins:

    Mena> d1 = d[u[1]:u[3]]

Since this assigns to a new variable, d1, the temporary() is not used.

Plotting Data

You must read and select data before you can plot anything.

This is important, and you may have jumped here without reading any of the top bits, so let me say it again:

You must read and select data before you can plot anything.

I know, that's not the way Ed code usually works. Ed code is usually self-sufficient, and can read data for you, then plot it.

However, reading data is a slow process, so for the time being, all my plotting routines get passed the data they will plot. This makes it really quick to experiment with the same dataset. It is likely that in the future, for production code, this mode of operation will be discontinued.