DasIO Struct Reference

Tracks input and output operations for das2 stream headers and data. More...

#include <das2/io.h>

Collaboration diagram for DasIO:

Public Member Functions
DAS_API DasIO *	new_DasIO_cfile (const char *sProg, FILE *file, const char *mode)
	Create a new DasIO object from a standard C FILE. More...
DAS_API DasErrCode	DasIO_model (DasIO *pThis, int nModel)
	Set the parsed stream data model. More...
DAS_API DasIO *	new_DasIO_cmd (const char *sProg, const char *sCmd)
	Create a new DasIO object from a shell command. More...
DAS_API DasIO *	new_DasIO_file (const char *sProg, const char *sFile, const char *mode)
	Create a new DasIO object from a disk file. More...
DAS_API DasIO *	new_DasIO_socket (const char *sProg, int nSockFd, const char *mode)
	Create a new DasIO object from a socket. More...
DAS_API DasIO *	new_DasIO_str (const char *sProg, char *sbuf, size_t len, const char *mode)
	Create a new DasIO object for reading/writing to memory buffer. More...
DAS_API DasIO *	new_DasIO_ssl (const char *sProg, void *pSsl, const char *mode)
	Create a new DasIO object using an encripted connection. More...
DAS_API int	DasIO_addProcessor (DasIO *pThis, StreamHandler *pProc)
	Add a packet processor to be invoked during I/O operations. More...
DAS_API int	DasIO_readAll (DasIO *pThis)
	Starts the processing of the stream read from FILE* infile. More...
DAS_API DasErrCode	DasIO_writeStreamDesc (DasIO *pThis, StreamDesc *pSd)
	Writes the data describing the stream to the output channel (e.g. More...
DAS_API DasErrCode	DasIO_writePktDesc (DasIO *pThis, PktDesc *pd)
	Writes the data describing a packet type to the output channel (e.g. More...
DAS_API DasErrCode	DasIO_writePktData (DasIO *pThis, PktDesc *pPd)
	Sends the data packet on to the stream after checking validity. More...
DAS_API DasErrCode	DasIO_writeException (DasIO *pThis, OobExcept *pSe)
	Output an exception structure.
DAS_API DasErrCode	DasIO_writeComment (DasIO *pThis, OobComment *pSc)
	Output a StreamComment Stream comments are generally messages interpreted only by humans and may change without affecting processes. More...
DAS_API DasErrCode	DasIO_sendLog (DasIO *pThis, int level, char *msg,...)
	Send a log message onto the stream at the given log level. More...
DAS_API void	DasIO_setLogLvl (DasIO *pThis, int minLevel)
	Set the minimum log level that will be transmitted on the stream. More...
DAS_API int	DasIO_getLogLvl (const DasIO *pThis)
	Get logging verbosity level. More...
DAS_API DasErrCode	DasIO_setTaskProgress (DasIO *pThis, int progress)
	Place rate-limited progress comments on an output stream. More...
DAS_API void	DasIO_throwException (DasIO *pThis, StreamDesc *pSd, const char *type, char *msg)
	Set the logging level. More...
DAS_API void	DasIO_close (DasIO *pThis)
	Normal stream close with no unusual condiditons Closes the output file descriptor, flushes a gzip buffer, etc. More...
DAS_API int	DasIO_printf (DasIO *pThis, const char *format,...)
	Print a string with a format specifier (Low-level API) This works similar to the C printf function. More...
DAS_API size_t	DasIO_write (DasIO *pThis, const char *data, int length)
	Anolog of fwrite (Low-level API)
DAS_API int	DasIO_read (DasIO *pThis, DasBuf *pBuf, size_t nBytes)
	Analog of fread (Low-level API)
DAS_API int	DasIO_readUntil (DasIO *pThis, DasBuf *pBuf, size_t nBytes, char cStop)
	Read until encountering a given byte (Low-level API) More...
DAS_API int	DasIO_getc (DasIO *pThis)
	Analog of getc (Low-level API)

Detailed Description

Tracks input and output operations for das2 stream headers and data.

Members of this class handle overall stream operations reading writing packets, checking packet lengths, passing XML string data off to descriptor object constructors, triggering processing callbacks and most other general Das2 Stream IO tasks.

Member Function Documentation

new_DasIO_cfile()

DAS_API DasIO * new_DasIO_cfile	(	const char *	sProg,
		FILE *	file,
		const char *	mode
	)

Create a new DasIO object from a standard C FILE.

Create a DasIO object that reads or writes to C standard IO files. The C file object's read or write state should be compatible with the DasIO state. For example:

DasIO* dio = new_DasIO_cfile(stdin, "myprogram", "rc");

is invalid, if standard input is not compressed, but the library has no way to tell until read operations start.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
file	a C standard IO file object.
mode	A string containing the packet tag mode, one of: 'r' read either tag type 'r2' read only das v2 packet tags (error on anything else) 'r3' read only das v3 packet tags (error on anything else) 'w','w2' write das v2 stream uncompressed 'w3' write das v3 stream uncompressed 'wc','wc2' write das v2 stream compressed 'wc3' write das v3 stream compressed

DasIO_model()

DAS_API DasErrCode DasIO_model	(	DasIO *	pThis,
		int	nModel
	)

Set the parsed stream data model.

When set to false either das2 or das3 data structs are gerenated depending on the stream content. When set to true, das2 data structures encountered in the stream are up-converted to das3.

Parameters

pThis	The DasIO object to configure, must be in read mode
nModel	The internal data sturcture version to use. If set to 2 any das3 structures encountered will trigger a failure. If set to 3 then any das2 structures will be upgraded to das3. Use -1 to indicate mixed model streams (not recommened)

Returns

DAS_OKAY if successful or an error code if not.

new_DasIO_cmd()

DAS_API DasIO * new_DasIO_cmd	(	const char *	sProg,
		const char *	sCmd
	)

Create a new DasIO object from a shell command.

Create a DasIO object that reads from a sub command. The sub command is run in the shell '/bin/sh' on POSIX systems

DasIO* dio = new_DasIO_cmd("/my/das2/reader 2017-01-01 2017-01-02", "wc");

is invalid, but the library has no way to tell until operations start.

Parameters

sProg	A spot to store the name of the program running the command this is useful for automatically generated error and log messages
sCmd	the command line to run, will be started via popen

new_DasIO_file()

DAS_API DasIO * new_DasIO_file	(	const char *	sProg,
		const char *	sFile,
		const char *	mode
	)

Create a new DasIO object from a disk file.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
sFile	the name of a file on disk. If the file doesn't exist it is created.
mode	A string containing the mode, one of: 'r' read (reads compressed and uncompressed files) 'w' write uncompressed 'wc' write compressed

new_DasIO_socket()

DAS_API DasIO * new_DasIO_socket	(	const char *	sProg,
		int	nSockFd,
		const char *	mode
	)

Create a new DasIO object from a socket.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
nSockFd	The socket file descriptor used for recv/write calls
mode	A string containing the mode, one of: 'r' read (reads compressed and uncompressed files) 'w' write uncompressed 'wc' write compressed

Returns

A new DasIO object allocated on the heap

new_DasIO_str()

DAS_API DasIO * new_DasIO_str	(	const char *	sProg,
		char *	sbuf,
		size_t	len,
		const char *	mode
	)

Create a new DasIO object for reading/writing to memory buffer.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
nSockFd	The socket file descriptor used for recv/write calls
mode	A string containing the mode, one of: 'r' read (reads compressed and uncompressed files) 'w' write uncompressed 'wc' write compressed

Returns

A new DasIO object allocated on the heap

new_DasIO_ssl()

DAS_API DasIO * new_DasIO_ssl	(	const char *	sProg,
		void *	pSsl,
		const char *	mode
	)

Create a new DasIO object using an encripted connection.

This class does not handle communications setup but can take over after a connection has been established and all needed headers have been sent and or read.

It is also assumed that the SSL connection has been established on a socket using BLOCKING I/O and is not sutiable for use as an action for a select() or poll() statement. To handle multiple connections at once in a single program create more than one DasIO object.

You should also setup the SSL connection with the SSL_MODE_AUTO_RETRY option to SSL_set_mode or SSL_CTX_set_mode to let the openssl library handle session renegotiation transparently. The http_getBodySocket() call does initialize any SSL structures it generates in auto-retry mode.

Parameters

sProg	A spot to store the name of the program creating the file this is useful for automatically generated error and log messages
pSsl	A pointer to OpenSSL SSL structure referencing an open connection that has been initialized in BLOCKING mode.
mode	A string containing the mode, one of: 'r' read (reads compressed and uncompressed files) 'w' write uncompressed 'wc' write compressed

Returns

DasIO_addProcessor()

DAS_API int DasIO_addProcessor	(	DasIO *	pThis,
		StreamHandler *	pProc
	)

Add a packet processor to be invoked during I/O operations.

A DasIO object may have 0 - DAS2_MAX_PROCESSORS packet processors attached to it. Each processor contains one or more callback functions that will be invoked and provided with DasDesc objects.

During stream read operations, processors are invoked after the header or data have been serialized from disk. During stream write operations processors are invoked before the write operation is completed.

Parameters

pThis	The DasIO object to receive the processor.
pProc	The StreamProc object to add to the list of processors.

Returns

The number of packet processors attached to this DasIO object or a negative error value if there is a problem.

DasIO_readAll()

DAS_API int DasIO_readAll

(

DasIO *

pThis

)

Starts the processing of the stream read from FILE* infile.

This function does not return until all input has been read or an exception condition has occurred. If a StreamHandler has been set with DasIO_setProcessor() then as each header packet and each data packet is read callbacks specified in the stream handler will be invoked. Otherwise all data is merely parsed and then discarded.

Returns

a status code. 0 indicates that all processing ended with no errors, otherwise a non-zero value is returned.

DasIO_writeStreamDesc()

DAS_API DasErrCode DasIO_writeStreamDesc	(	DasIO *	pThis,
		StreamDesc *	pSd
	)

Writes the data describing the stream to the output channel (e.g.

File* ).

This serializes the descriptor structure into XML and writes it out.

Parameters

pThis	The IO object, must be set up in a write mode
pSd	The stream descriptor to serialize

Returns

0 on success a positive integer error code other wise.

DasIO_writePktDesc()

DAS_API DasErrCode DasIO_writePktDesc	(	DasIO *	pThis,
		PktDesc *	pd
	)

Writes the data describing a packet type to the output channel (e.g.

File* ).

This serializes the descriptor structure into XML and writes it out. The packet type will be assigned a number on the das2Stream, and data packets will be tagged with this number.

DasIO_writePktData()

DAS_API DasErrCode DasIO_writePktData	(	DasIO *	pThis,
		PktDesc *	pPd
	)

Sends the data packet on to the stream after checking validity.

This check insures that all planes have been set via setDataPacket.

Parameters

pThis	the IO object to use for sending the data
pPd	a Packet Descriptor loaded with values for output.

Returns

0 if the operation was successful or an positive value on an error.

DasIO_writeComment()

DAS_API DasErrCode DasIO_writeComment	(	DasIO *	pThis,
		OobComment *	pSc
	)

Output a StreamComment Stream comments are generally messages interpreted only by humans and may change without affecting processes.

Progress messages are sent via stream comments.

DasIO_sendLog()

DAS_API DasErrCode DasIO_sendLog	(	DasIO *	pThis,
		int	level,
		char *	msg,
			...
	)

Send a log message onto the stream at the given log level.

Note that messages sent at a level greater than INFO may be thrown out if performance would be degraded.

DasIO_setLogLvl()

DAS_API void DasIO_setLogLvl	(	DasIO *	pThis,
		int	minLevel
	)

Set the minimum log level that will be transmitted on the stream.

The point of the DasIO logging functions is not to handle debugging, but to inform a client program what is going on by embedding messages in an output stream. Since the point of a stream is to transmit data, placing too many comments in the stream will degrade performance.

Unless this function is called, the default logging level is LOGLEVEL_WARNING

Parameters

pThis	the DasIO object who's log level will be altered.
minLevel	A logging level, one of: LOGLVL_ERROR - Only output messages that indicate improper operation. LOGLVL_WARNING - Messages that indicate a problem but processing can continue LOGLVL_INFO - Extra stuff the operator may want to know LOGLVL_CONFIG - Include basic setup information as well

DasIO_getLogLvl()

DAS_API int DasIO_getLogLvl

(

const DasIO *

pThis

)

Get logging verbosity level.

Parameters

pThis

a DasIO object to query

Returns

A logging level as defined in DasIO_setLogLevel()

DasIO_setTaskProgress()

DAS_API DasErrCode DasIO_setTaskProgress	(	DasIO *	pThis,
		int	progress
	)

Place rate-limited progress comments on an output stream.

Messages are decimated dynamically to try to limit the stream comment rate to about 10 per second (see targetUpdateRateMilli). If Note the tacit assumption that is the stream is reread, it will be reread at a faster rate than it was written. Processes reducing a stream should take care to call setTaskProgress themselves instead of simply forwarding the progress comments, so that the new stream is not burdened by excessive comments. See setTaskSize().

DasIO_throwException()

DAS_API void DasIO_throwException	(	DasIO *	pThis,
		StreamDesc *	pSd,
		const char *	type,
		char *	msg
	)

Set the logging level.

Close and free an output stream with an exception.

Call this to notify stream consumers that there was an exceptional condition that prevents completion of the stream. For example, this might be used to indicate that no input files were available. This is a convenience function for the calls:

• DasIO_writeStreamDesc()
• DasIO_writeException()
• DasIO_close()
• del_DasIO()

Parameters

pThis	the output object to receive the exception packet
pSd	The stream descriptor. All Das2 Streams have to start with a stream header. If this header hasn't been output then this object will be serialized to create a stream header.
type	the type of exception may be one of the pre-defined strings: DAS2_EXCEPT_NO_DATA_IN_INTERVAL DAS2_SERVER_ERROR or some other short type string of your choice.
msg	a longer message explaining what went wrong.

DasIO_close()

DAS_API void DasIO_close

(

DasIO *

pThis

)

Normal stream close with no unusual condiditons Closes the output file descriptor, flushes a gzip buffer, etc.

May possibly send a stream termination comment as well.

Parameters

pThis

DasIO_printf()

DAS_API int DasIO_printf	(	DasIO *	pThis,
		const char *	format,
			...
	)

Print a string with a format specifier (Low-level API) This works similar to the C printf function.

Parameters

pThis
format
...	The variables to print

Returns

The number of characters printed.

DasIO_readUntil()

DAS_API int DasIO_readUntil	(	DasIO *	pThis,
		DasBuf *	pBuf,
		size_t	nBytes,
		char	cStop
	)

Read until encountering a given byte (Low-level API)

Read until hitting the stop byte. The stop byte is copied to the buffer.

---

The documentation for this struct was generated from the following file:

• das2/io.h