DasIO Struct Reference

#include <das2/das2io.h>

Detailed Description

Tracks input and output operations for das2 stream headers and data.

Members of this class handle overall stream operations reading writing packets, checking packet lengths, passing XML string data off to descriptor object constructors, triggering processing callbacks and most other general Das2 Stream IO tasks.

Public Member Functions

DasIO * new_DasIO_cfile (const char *sProg, FILE *file, const char *mode)
 Create a new DasIO object from a standard C FILE.
DasIO * new_DasIO_file (const char *sProg, const char *sFile, const char *mode)
 Create a new DasIO object from a disk file.
int DasIO_addProcessor (DasIO *pThis, StreamHandler *pProc)
 Add a packet processor to be invoked during I/O operations.
int DasIO_readAll (DasIO *pThis)
 Starts the processing of the stream read from FILE* infile.
ErrorCode DasIO_writeStreamDesc (DasIO *pThis, StreamDesc *pSd)
 Writes the data describing the stream to the output channel (e.g.
ErrorCode DasIO_writePktDesc (DasIO *pThis, PktDesc *pd)
 Writes the data describing a packet type to the output channel (e.g.
ErrorCode DasIO_writePktData (DasIO *pThis, PktDesc *pPd)
 Sends the data packet on to the stream after checking validity.
ErrorCode DasIO_writeException (DasIO *pThis, OobExcept *pSe)
 Output an exception structure.
ErrorCode DasIO_writeComment (DasIO *pThis, OobComment *pSc)
 Output a StreamComment Stream comments are generally messages interpreted only by humans and may change without affecting processes.
ErrorCode DasIO_sendLog (DasIO *pThis, int level, char *msg,...)
 Send a log message onto the stream at the given log level.
void DasIO_setLogLvl (DasIO *pThis, int minLevel)
 Set the minimum log level that will be transmitted on the stream.
int DasIO_getLogLvl (const DasIO *pThis)
 Get logging verbosity level.
ErrorCode DasIO_setTaskProgress (DasIO *pThis, int progress)
 Place rate-limited progress comments on an output stream.
void DasIO_throwException (DasIO *pThis, StreamDesc *pSd, const char *type, char *msg)
 Set the logging level.
void DasIO_close (DasIO *pThis)
 Analog of fclose Closes the output file descriptor, flushes a gzip buffer, etc.
int DasIO_printf (DasIO *pThis, const char *format,...)
 Print a string with a format specifier (Low-level API) This works similar to the C printf function.
int DasIO_write (DasIO *pThis, const char *data, int length)
 Anolog of fwrite (Low-level API).
int DasIO_read (DasIO *pThis, DasBuf *pBuf, size_t nBytes)
 Analog of fread (Low-level API).
int DasIO_getc (DasIO *pThis)
 Analog of getc (Low-level API).

Member Function Documentation

DasIO * new_DasIO_cfile ( const char *  sProg,
FILE *  file,
const char *  mode 
)

Create a new DasIO object from a standard C FILE.

Create a DasIO object that reads or writes to C standard IO files. The C file object's read or write state should be compatible with the DasIO state. For example: 

  DasIO* dio = new_DasIO(stdin, "wc");

is invalid, but the library has no way to tell until operations start.

Parameters:
file a C standard IO file object.
mode A string containing the mode, one of:

  • 'r' read
  • 'w' write uncompressed
  • 'wc' write compressed
DasIO * new_DasIO_file ( const char *  sProg,
const char *  sFile,
const char *  mode 
)

Create a new DasIO object from a disk file.

Parameters:
sFile the name of a file on disk. If the file doesn't exist it is created.
mode A string containing the mode, one of:

  • 'r' read (reads compressed and uncompressed files)
  • 'w' write uncompressed
  • 'wc' write compressed
int DasIO_addProcessor ( DasIO *  pThis,
StreamHandler *  pProc 
)

Add a packet processor to be invoked during I/O operations.

A DasIO object may have 0 - DAS2_MAX_PROCESSORS packet processors attached to it. Each processor contains one or more callback functions that will be invoked and provided with Descriptor objects.

During stream read operations, processors are invoked after the header or data have been serialized from disk. During stream write operations processors are invoked before the write operation is completed.

Parameters:
pThis The DasIO object to receive the processor.
pProc The StreamProc object to add to the list of processors.
Returns:
The number of packet processors attached to this DasIO object or a negative error value if there is a problem.
int DasIO_readAll ( DasIO *  pThis  ) 

Starts the processing of the stream read from FILE* infile.

This function does not return until all input has been read or an exception condition has occurred. If a StreamHandler has been set with DasIO_setProcessor() then as each header packet and each data packet is read callbacks specified in the stream handler will be invoked. Otherwise all data is merely parsed and then discarded.

Returns:
a status code. 0 indicates that all processing ended with no errors, otherwise a non-zero value is returned.
ErrorCode DasIO_writeStreamDesc ( DasIO *  pThis,
StreamDesc *  pSd 
)

Writes the data describing the stream to the output channel (e.g.

File* ).

This serializes the descriptor structure into XML and writes it out.

Parameters:
pThis The IO object, must be set up in a write mode
pSd The stream descriptor to serialize
Returns:
0 on success a positive integer error code other wise.
ErrorCode DasIO_writePktDesc ( DasIO *  pThis,
PktDesc *  pd 
)

Writes the data describing a packet type to the output channel (e.g.

File* ).

This serializes the descriptor structure into XML and writes it out. The packet type will be assigned a number on the das2Stream, and data packets will be tagged with this number.

ErrorCode DasIO_writePktData ( DasIO *  pThis,
PktDesc *  pPd 
)

Sends the data packet on to the stream after checking validity.

This check insures that all planes have been set via setDataPacket.

the IO object to use for sending the data a Packet Descriptor loaded with values for output.

Returns:
0 if the operation was successful or an positive value on an error.
ErrorCode DasIO_writeComment ( DasIO *  pThis,
OobComment *  pSc 
)

Output a StreamComment Stream comments are generally messages interpreted only by humans and may change without affecting processes.

Progress messages are sent via stream comments.

ErrorCode DasIO_sendLog ( DasIO *  pThis,
int  level,
char *  msg,
  ... 
)

Send a log message onto the stream at the given log level.

Note that messages sent at a level greater than INFO may be thrown out if performance would be degraded.

void DasIO_setLogLvl ( DasIO *  pThis,
int  minLevel 
)

Set the minimum log level that will be transmitted on the stream.

The point of the DasIO logging functions is not to handle debugging, but to inform a client program what is going on by embedding messages in an output stream. Since the point of a stream is to transmit data, placing too many comments in the stream will degrade performance.

Unless this function is called, the default logging level is LOGLEVEL_WARNING

Parameters:
@param minLevel A logging level, one of:

  • LOGLVL_ERROR - Only output messages that indicate improper operation.
  • LOGLVL_WARNING - Messages that indicate a problem but processing can continue
  • LOGLVL_INFO - Extra stuff the operator may want to know
  • LOGLVL_CONFIG - Include basic setup information as well
int DasIO_getLogLvl ( const DasIO *  pThis  ) 

Get logging verbosity level.

Parameters:
pThis a DasIO object to query
Returns:
A logging level as defined in DasIO_setLogLevel()
ErrorCode DasIO_setTaskProgress ( DasIO *  pThis,
int  progress 
)

Place rate-limited progress comments on an output stream.

Messages are decimated dynamically to try to limit the stream comment rate to about 10 per second (see targetUpdateRateMilli). If Note the tacit assumption that is the stream is reread, it will be reread at a faster rate than it was written. Processes reducing a stream should take care to call setTaskProgress themselves instead of simply forwarding the progress comments, so that the new stream is not burdened by excessive comments. See setTaskSize().

void DasIO_throwException ( DasIO *  pThis,
StreamDesc *  pSd,
const char *  type,
char *  msg 
)

Set the logging level.

Close and free an output stream with an exception.

Call this to notify stream consumers that there was an exceptional condition that prevents completion of the stream. For example, this might be used to indicate that no input files were available. This is a convenience function for the calls:

Parameters:
pThis the output object to receive the exception packet
pSd The stream descriptor. All Das2 Streams have to start with a stream header. If this header hasn't been output then this object will be serialized to create a stream header.
type the type of exception may be one of the pre-defined strings:

  • DAS2_EXCEPT_NO_DATA_IN_INTERVAL
  • DAS2_SERVER_ERROR or some other short type string of your choice.
msg a longer message explaining what went wrong.
void DasIO_close ( DasIO *  pThis  ) 

Analog of fclose Closes the output file descriptor, flushes a gzip buffer, etc.

May possibly send a stream termination comment as well.

Parameters:
pThis 
int DasIO_printf ( DasIO *  pThis,
const char *  format,
  ... 
)

Print a string with a format specifier (Low-level API) This works similar to the C printf function.

Parameters:
pThis 
format 
... The variables to print
Returns:
The number of characters printed.
The documentation for this struct was generated from the following file:
 All Data Structures Files Functions Variables Typedefs Enumerations Defines
Generated on 11 Sep 2016 for Das2 Stream Utilities by  doxygen 1.6.1