JLS Format

group jls_format

JLS file format.

Defines

JLS_FORMAT_VERSION_MAJOR

JLS_FORMAT_VERSION_MINOR

JLS_FORMAT_VERSION_PATCH

JLS_FORMAT_VERSION_U32

JLS_HEADER_IDENTIFICATION

The file identification bytes at the start of the file.

These bytes are not arbitrary. We carefully selected them to provide:

Identification: Help the application determine that this file is in the correct format with minimal uncertainty.
Correct endianness: Little endian has won, so this entire format is stored in little endian format.
Proper binary processing: The different line ending combinations ensure that the reader is not “fixing” the line endings, since this is a binary file format.
Display: Include “substitute” and “file separator” so that text printers to not show the rest of the file.

The following table describes maps each byte to its purpose:

Value (hex)	Purpose
6A 6C 73 66 6d 74	ASCII “jlsfmt”, when viewed in text editor.
0D 0A	DOS line ending to ensure binary correctness.
20	ASCII space.
0A	UNIX line ending to ensure binary correctness.
20	ASCII space.
1A	Substitute character (stops listing under Windows).
20 20	ASCII spaces.
B2	Ensure that system supports 8-bit data
1C	File separator.

JLS_SOURCE_COUNT: The maximum allowed number of sources.

JLS_SIGNAL_COUNT: The maximum allowed number of signals.

JLS_SUMMARY_LEVEL_COUNT: The number of summary levels.

JLS_TRACK_TAG_FLAG

JLS_TRACK_TAG_PACK(track_type, track_chunk)

Pack the chunk tag.

Parameters:

track_type – The jls_track_type_e
track_chunk – The jls_track_chunk_e

Returns:

The tag value.

JLS_TRACK_TAG_PACKER(track_type, track_chunk)

JLS_DATATYPE_BASETYPE_INT

JLS_DATATYPE_BASETYPE_UNSIGNED

JLS_DATATYPE_BASETYPE_UINT

JLS_DATATYPE_BASETYPE_FLOAT

JLS_DATATYPE_DEF(basetype, size, q)

Construct a JLS datatype.

Parameters:

basetype – The datatype base type, one of [INT, UINT, FLOAT, BOOL]
size – The size in bits. Only the following options are supported:
- INT: 4, 8, 16, 24, 32, 64
- UINT: 1, 4, 8, 16, 24, 32, 64
- FLOAT: 32, 64
- BOOL = UINT 1
q – The signed fixed-point location, only valid for INT and UINT. Set to 0 for normal, whole numbers. Set to 0 for FLOAT and BOOL. The fixed point is between bits q and q-1. The value is scaled by 2 ** -q.

JLS_DATATYPE_I4

JLS_DATATYPE_I8

JLS_DATATYPE_I16

JLS_DATATYPE_I24

JLS_DATATYPE_I32

JLS_DATATYPE_I64

JLS_DATATYPE_U1

JLS_DATATYPE_U4

JLS_DATATYPE_U8

JLS_DATATYPE_U16

JLS_DATATYPE_U24

JLS_DATATYPE_U32

JLS_DATATYPE_U64

JLS_DATATYPE_BOOL

JLS_DATATYPE_F32

JLS_DATATYPE_F64

Enums

enum jls_signal_type_e

The signal type definition.

Values:

enumerator JLS_SIGNAL_TYPE_FSR: Fixed sampling rate.

enumerator JLS_SIGNAL_TYPE_VSR: Variable sampling rate.

enum jls_track_type_e

The available track types that store data over time.

Values:

enumerator JLS_TRACK_TYPE_FSR

Block tracks contain fixed sample-rate (FSR) data.

The JLS_TAG_SIGNAL_DEF defines the sampling rate.

enumerator JLS_TRACK_TYPE_VSR

Fixed-type, variable-sample-rate (VSR) time series data.

Each data entry consists of time in UTC and the data.

enumerator JLS_TRACK_TYPE_ANNOTATION

Annotations contain infrequent, variable-typed data.

Each annotation data entry consists of time and associated annotation data. For FSR, time must be samples_id. For VSR, time must be UTC.

See also

jls_annotation_type_e for the annotation types.

enumerator JLS_TRACK_TYPE_UTC

The UTC track associates sample_id with UTC.

Each utc data entry consists of sample_id, UTC timestamp pairs. This track is only used for FSR signals.

enumerator JLS_TRACK_TYPE_COUNT: The total number of track types.

enum jls_storage_type_e

The data storage type.

The data storage type applies to directly user-accessible data including annotations and user_data.

Values:

enumerator JLS_STORAGE_TYPE_INVALID: Invalid (unknown) storage type.

enumerator JLS_STORAGE_TYPE_BINARY: Raw binary data.

enumerator JLS_STORAGE_TYPE_STRING: Null-terminated C-style string with UTF-8 encoding.

enumerator JLS_STORAGE_TYPE_JSON: JSON serialized data structure with NULL terminator and UTF-8 encoding.

enum jls_annotation_type_e

The available annotation types.

Values:

enumerator JLS_ANNOTATION_TYPE_USER

Arbitrary user data.

Application-dependent data with no standardized form or purpose.

enumerator JLS_ANNOTATION_TYPE_TEXT

UTF-8 formatted text.

Viewers should display this text at the appropriate location. The jls_storage_type_e must be STRING.

enumerator JLS_ANNOTATION_TYPE_VERTICAL_MARKER

A vertical marker at a given time.

Marker names can be arbitrary, but the convention is:

Number strings, like “1”, represent a single marker.
Alpha + number string, like “A1” and “A2”, represent a marker pair (dual markers). The jls_storage_type_e must be STRING.

enumerator JLS_ANNOTATION_TYPE_HORIZONTAL_MARKER

A horizontal marker at a given y-axis value.

Marker names can be arbitrary, but the convention is:

Number strings, like “1”, represent a single marker.
Alpha + number string, like “A1” and “A2”, represent a marker pair (dual markers). The jls_storage_type_e must be STRING.

enum jls_track_chunk_e

The chunks used to store the track information.

Values:

enumerator JLS_TRACK_CHUNK_DEF

Track definition chunk.

This chunk contains a zero-length (empty) payload.

enumerator JLS_TRACK_CHUNK_HEAD

Track offsets for the first chunk at each level.

This chunk provides fast seek access to the first chunk at each summary level for this track. The payload is jls_track_head_s which contains the offset to the first INDEX chunk.

See also

jls_track_head_s for all track types.

enumerator JLS_TRACK_CHUNK_DATA

The data chunk.

The payload varies by track type and data format. All DATA payloads start with jls_payload_header_s.

See also

jls_fsr_f32_data_s for FSR float32.

See also

jls_annotation_s for ANNOTATION.

See also

jls_utc_data_s for UTC.

enumerator JLS_TRACK_CHUNK_INDEX

Provides the timestamp and offset for each contributing data chunk.

This chunk MUST be immediately follow by a SUMMARY chunk. All INDEX payloads start with jls_payload_header_s.

See also

jls_fsr_index_s for FSR track types.

See also

jls_index_s for all other track types.

enumerator JLS_TRACK_CHUNK_SUMMARY

The summary chunk.

The payload format is defined by the track type. All CHUNK payloads start with jls_payload_header_s.

See also

jls_fsr_f32_summary_s for FSR float32.

See also

jls_annotation_summary_s for ANNOTATION.

See also

jls_utc_summary_s for UTC.

enum jls_tag_e

The tag definitions.

Values:

enumerator JLS_TAG_INVALID

enumerator JLS_TAG_SOURCE_DEF

enumerator JLS_TAG_SIGNAL_DEF

enumerator JLS_TAG_TRACK_FSR_DEF

enumerator JLS_TAG_TRACK_FSR_HEAD

enumerator JLS_TAG_TRACK_FSR_DATA

enumerator JLS_TAG_TRACK_FSR_INDEX

enumerator JLS_TAG_TRACK_FSR_SUMMARY

enumerator JLS_TAG_TRACK_VSR_DEF

enumerator JLS_TAG_TRACK_VSR_HEAD

enumerator JLS_TAG_TRACK_VSR_DATA

enumerator JLS_TAG_TRACK_VSR_INDEX

enumerator JLS_TAG_TRACK_VSR_SUMMARY

enumerator JLS_TAG_TRACK_ANNOTATION_DEF

enumerator JLS_TAG_TRACK_ANNOTATION_HEAD

enumerator JLS_TAG_TRACK_ANNOTATION_DATA

enumerator JLS_TAG_TRACK_ANNOTATION_INDEX

enumerator JLS_TAG_TRACK_ANNOTATION_SUMMARY

enumerator JLS_TAG_TRACK_UTC_DEF

enumerator JLS_TAG_TRACK_UTC_HEAD

enumerator JLS_TAG_TRACK_UTC_DATA

enumerator JLS_TAG_TRACK_UTC_INDEX

enumerator JLS_TAG_TRACK_UTC_SUMMARY

enumerator JLS_TAG_USER_DATA

enumerator JLS_TAG_END

enum jls_summary_fsr_e

The summary storage order for each entry.

Values:

enumerator JLS_SUMMARY_FSR_MEAN

enumerator JLS_SUMMARY_FSR_STD

enumerator JLS_SUMMARY_FSR_MIN

enumerator JLS_SUMMARY_FSR_MAX

enumerator JLS_SUMMARY_FSR_COUNT

Functions

static inline uint8_t jls_datatype_parse_basetype(uint32_t dt)

static inline uint8_t jls_datatype_parse_size(uint32_t dt)

static inline uint8_t jls_datatype_parse_q(uint32_t dt)

struct jls_source_def_s

#include <format.h>

The source definition.

Public Members

uint16_t source_id

The source identifier.

0 reserved for global annotations, must be unique per instance

const char *name: The source name string.

const char *vendor: The vendor name string.

const char *model: The model string.

const char *version: The version string.

const char *serial_number: The serial number string.

struct jls_signal_def_s

#include <format.h>

The signal definition.

Public Members

uint16_t signal_id

The signal identifier.

0 to JLS_SIGNAL_COUNT - 1, must be unique per instance

uint16_t source_id: The source identifier, must match a source_def.

uint8_t signal_type: The jls_signal_type_e signal type.

uint32_t data_type: The JLS_DATATYPE_* data type for this signal.

uint32_t sample_rate: TThe sample rate per second (Hz). 0 for VSR.

uint32_t samples_per_data: The number of samples per data chunk. (write suggestion)

uint32_t sample_decimate_factor: The number of samples per summary level 1 entry.

uint32_t entries_per_summary: The number of entries per summary chunk. (write suggestion)

uint32_t summary_decimate_factor: The number of summaries per summary, level >= 2.

uint32_t annotation_decimate_factor: The annotation decimate factor for summaries.

uint32_t utc_decimate_factor: The UTC decimate factor for summaries.

int64_t sample_id_offset: The sample id offset for the first sample. (FSR only)

const char *name: The signal name.

const char *units: The units string, normally as SI with no scale prefix.

struct jls_track_head_s: #include <format.h>

The track head payload for JLS_TRACK_CHUNK_HEAD.

union jls_version_u

#include <format.h>

Union structure for parsing 32-bit versions.

Public Members

uint32_t u32

uint16_t patch

uint8_t minor

uint8_t major

struct jls_version_u::[anonymous] s

struct jls_file_header_s

#include <format.h>

The JLS file header structure.

Public Members

uint8_t identification[16]

The semi-unique file identification header.

See also

JLS_HEADER_IDENTIFICATION

uint64_t length

The file length in bytes.

This is the last field updated on file close. If the file was not closed gracefully, the value will be 0.

union jls_version_u version

The JLS file format version number.

See also

JLS_FORMAT_VERSION_U32

uint32_t crc32: The CRC32 from the start of the file through version.

struct jls_chunk_header_s

#include <format.h>

The JLS chunk header structure.

Every chunk starts with this header. This header identifies the chunk’s payload, and enables the chunk to participate in one doubly-linked list.

If the length is zero, then the next chuck header immediately follows this chunk header. If the length is not zero, then the chunk consists of:

A chunk header
payload of length bytes
Zero padding of 0-7 bytes, so that the entire chunk will end on a multiple of 8 bytes. This field ends on: 8 * k - 4
crc32 over the payload.

Public Members

uint64_t item_next

The next item.

The chunk header enables each chunk to participate in a single, doubly-linked list. This field indicates the location for the next item in the list. The value is relative to start of the file. 0 indicates end of list.

This field allows simple, linear traversal of data, but the next chunk is not known when the chunk is first created. Therefore, this field requires that chunk headers are updated when the software writes the next item to the file.

uint64_t item_prev

The previous item.

The chunk header enables each chunk to participate in a single, doubly-linked list. This field indicates the location for the previous item in the list. The value is relative to start of the file. 0 indicates start of list.

uint8_t tag

The tag.

The jls_tag_e value that identifies the contents of this chunk.

uint8_t rsv0_u8: Reserved for future use. (compression?)

uint16_t chunk_meta

The metadata associated with this chunk.

Each tag is free to define the purpose of this field.

However, all data tags use this definition:

chunk_meta[7:0] is the signal/time series identifier from 0 to 255.
chunk_meta[11:8] is reserved.
chunk_meta[15:12] contains the depth for this chunk from 0 to 15.
- 0 = block (sample level)
- 1 = First-level summary of block samples
- 2 = Second-level summary of first-level summaries.

User-data reserves chunk_meta[15:12] to store the storage_type and another internal indications. chunk_meta[11:0] may be assigned by the specific application.

uint32_t payload_length

The length of the payload in bytes.

Can be 0.

In addition to defining the payload size, this value is also used for forward chunk traversal.

uint32_t payload_prev_length

The length of the previous payload in bytes.

Used for reverse chunk traversal.

uint32_t crc32: The CRC32 over the header, excluding this field.

struct jls_payload_header_s

#include <format.h>

The payload header for DATA, INDEX, and SUMMARY chunks.

Public Members

int64_t timestamp: The sample_id for the first entry.

uint32_t entry_count: The total number of entries.

uint16_t entry_size_bits: The size of each entry, in bits.

uint16_t rsv16: Reserved.

struct jls_fsr_data_s

#include <format.h>

The FSR data chunk format.

Public Members

struct jls_payload_header_s header: The payload header.

float data[]

The summary data.

Although data’s type is float, the actual data type depends upon the signal definition.

struct jls_fsr_index_s

#include <format.h>

The payload for JLS_TAG_TRACK_FSR_INDEX chunks.

Since FSR has a fixed sample rate, the header contains enough information to fully identify the timestamp for each offset. Therefore, no additional time information is required per entry.

See also

jls_index_s for all other INDEX chunk types.

Public Members

uint64_t offsets[]: The chunk file offsets, spaced by fixed time intervals.

struct jls_fsr_f32_summary_s

#include <format.h>

The FSR summary chunk format with f32 values.

This summary format is used by all types except u32, u64, i32, i64, f64.

Public Members

struct jls_payload_header_s header: The payload.

float data[][JLS_SUMMARY_FSR_COUNT]: The summary data, each entry is 4 x f32: mean, std, min, max.

struct jls_fsr_f64_summary_s

#include <format.h>

The FSR summary chunk format with f64 values.

This summary format is used by u64, i64, f64.

Public Members

struct jls_payload_header_s header: The payload.

double data[][JLS_SUMMARY_FSR_COUNT]: The summary data, each entry is 4 x f64: mean, std, min, max.

struct jls_index_entry_s

#include <format.h>

The entry format for JLS_TRACK_CHUNK_INDEX payloads.

See also

jls_index_s

Public Members

int64_t timestamp: The timestamp for this entry. sample_id for FSR, UTC for VSR.

uint64_t offset: The chunk file offset.

struct jls_index_s

#include <format.h>

The payload for JLS_TRACK_CHUNK_INDEX chunks.

The INDEX payload maps timestamps to offsets to allow fast seek. However, the JLS_TAG_TRACK_FSR_INDEX uses the jls_fsr_index_s since the timestamp provides unnecessary, duplicative information for FSR tracks.

See also

jls_fsr_index_s for JLS_TAG_TRACK_FSR_INDEX.

struct jls_annotation_s

#include <format.h>

Hold a single annotation record.

This structure is used by both the API and JLS_TAG_TRACK_ANNOTATION_DATA.

Public Members

int64_t timestamp: The timestamp for this annotation. sample_id for FSR, UTC for VSR.

uint64_t rsv64_1: Reserved, write to 0.

uint8_t annotation_type: The jls_annotation_type_e.

uint8_t storage_type: The jls_storage_type_e.

uint8_t group_id: The optional group identifier. If unused, write to 0.

uint8_t rsv8_1: Reserved, write to 0.

float y: The y-axis value or NAN to automatically position.

uint32_t data_size: The size of data in bytes.

uint8_t data[]: The annotation data.

struct jls_annotation_summary_entry_s

#include <format.h>

The entry format for JLS_TAG_TRACK_ANNOTATION_SUMMARY.

See also

jls_annotation_summary_s

Public Members

int64_t timestamp: The timestamp (duplicates INDEX).

uint8_t annotation_type: The jls_annotation_s.annotation_type.

uint8_t group_id: The jls_annotation_s.group_id.

uint8_t rsv8_1: Reserved, write to 0.

uint8_t rsv8_2: Reserved, write to 0.

float y: The jls_annotation_s.y.

struct jls_annotation_summary_s: #include <format.h>

The payload format for JLS_TAG_TRACK_ANNOTATION_SUMMARY chunks.

struct jls_utc_data_s

#include <format.h>

The entry format for JLS_TAG_TRACK_UTC_DATA.

This same format is reused for summary entries. UTC.DATA only exists to provide recovery in the event that the file is not properly closed.

See also

jls_utc_summary_s

Public Members

int64_t timestamp: The timestamp in UTC.

struct jls_utc_summary_entry_s

#include <format.h>

The entry format for JLS_TAG_TRACK_UTC_SUMMARY.

This same format is reused for summary entries. UTC.DATA only exists to provide recovery in the event that the file is not properly closed.

See also

jls_utc_summary_s

Public Members

int64_t sample_id: The timestamp in sample ids (duplicates INDEX).

int64_t timestamp: The timestamp in UTC.

struct jls_utc_summary_s: #include <format.h>

The payload format for JLS_TAG_TRACK_UTC_SUMMARY chunks.