GAPLESS DATA STREAMING
Many user applications require gapless signal acquisition and simultaneous storage or analysis of that data. These include satellite signal monitoring, electronic surveillance, imaging etc.
Gapless data acquisition applications can be divided into two types:
1. Gapless acquisition for real-time analysis
2. Gapless acquisition for storage, also known as Gapless Streaming Systems
AlazarTech serves both types of gapless acquisition applications. In this discussion, we will shed some more light on Gapless Streaming Systems.
By definition, a Gapless Streaming System needs a mass storage medium to save the acquired data in. There are many types of media utilized by customers based on their specific storage speed and capacity requirements.
STORAGE IN ON-BOARD MEMORY
Almost all AlazarTech products have on-board memory. On the more advanced models, this can be as high as 8 Gigabytes (GB). For most applications, this on-board memory is used as a very deep FIFO between the Analog-to-Digital Converters (ADCs) and PCI Express (PCIe) bus, thereby providing protection against operating system latencies. However, the same memory can also be used as high speed storage for later retrieval over PCIe bus, as long as the total amount of storage required is less than the amount of on-board memory.
Pros: Very fast data storage, independent of bus speed
Cons: Limited memory size
STORAGE IN MOTHERBOARD MEMORY
As mentioned above, on-board memory can be used as a very deep FIFO to stream data across the bus to a user buffer.
It should be noted that some other manufacturers claim to have streaming capability, but can only stream to buffers in kernel memory that require CPU supervised copying to user buffers. This results in much slower overall throughput. AlazarTech, on the other hand, can stream to user buffers directly using DMA.
One advantage of being able to stream data to user buffers is that users can allocate very large buffers (> 50GB in a computer with 64GB RAM) and stream to this memory. This is a very low cost way of acquiring tens of Gigabytes of data.
AlazarDSO’s Stream to Memory feature does exactly this. AlazarDSO allocates the biggest buffer the operating system allows it to allocate and then streams data at the full bus speed to this buffer. Once the buffer has been filled up, AlazarDSO copies the contents of this buffer to a hard disk for later analysis. Note that storage to the hard disk at the end takes place at the speed supported by the hard disk, which may be very slow depending on the system. The main point is that AlazarDSO allows storage of tens of Gigabytes of data into motherboard memory at very high speeds.
ATS-SDK also provides C example programs of how to acquire gapless data to motherboard memory under programmatic control.
Memory capacity of motherboards is on the increase. 192 GB RAM capacity is almost mainstream, whereas 1TB RAM is possible on high end servers and workstations, albeit very expensive.
Pros: Very fast data storage at full bus speed at low cost for 10’s of GB of data storage.
Cons: Memory size imited by motherboard memory. Very expensive beyond 50 GB storage capacity.
STORAGE IN PCIE SSD CARDS
Another popular storage medium for gapless data is a PCIe SSD card made by vendors such as Intel, Micron etc.
These are PCI Express cards that plug into an adjacent slot and appear as a disk drive to the operating system. Since they do not have to go through SATA or SAS interface, these disk drives are much faster. Also, they have no moving parts so they are more reliable in the long run.
PCIe SSDs typically have storage capacity of 512 GB or even 1 TB. Larger devices will be available in the future.
Unfortunately, to date, PCIe SSDs do not support full bus bandwidth for average sequential write speed and are limited to approximately 1.8 GB/s. We expect this speed to increase in the future.
Pros: Relatively fast throughput, no moving parts, small physical size.
Cons: Limited capacity, expensive for higher capacities, do not support full bus speed for average write speed.
STORAGE IN RAID SYSTEMS
The highest performance gapless streaming systems are based on Redundant Array of Independent Disks (RAID). There are many levels of RAID, but the most popular for gapless streaming applications are RAID0 and RAID5.
The basic idea behind RAID is to parallelize data storage across multiple disks. RAID0 (also known as striping) simply writes a packet of data to each disk simultaneously, so if each disk has an average sequential write speed of 100 MB/s, eight such disks should result in average write speed of 800 MB/s. Of course, nothing is quite as simple as this, but this is how RAID0 works.
RAID5 adds some redundancy to data storage, i.e. if one of the disks in the RAID array goes bad, it is possible to recover the data (within certain limits of course).
It should be noted that the average write speed of a streaming system is determined by the slowest component in the chain:
• A/D conversion rate
• PCI Bus throughput
• RAID controller throughput
• RAID system’s average write speed
THE ALAZARTECH ADVANTAGE
AlazarTech has developed AlazarStream systems that provide gapless storage of A/D data to RAID volumes. AlazarStream is a complete computer system with a RAID volume and motherboard slots for AlazarTech waveform digitizer boards. All components are carefully selected for interoperability and each system is fully tested and characterized for average write speed.
There are two types of AlazarStream systems:
• SSD based
• HDD based
SSD based AlazarStream systems, such as AlazarStream 3000, provide the highest performance. Average write speed in excess of 3000MB/sec (3 GB/s) has been demonstrated with RAID0. This system consists of a 2U rackmountable chassis, embedded RAID controller, 24 SSDs, a high end dual CPU server class motherboard running Windows Server operating system and redundant power supply.
The maximum capacity of this solution is 20 TB.
HDD based AlazarSream systems, such as AlazarStream 2000, provide slightly lower average write speed, but can have much larger storage capacity. Average write speed of 2000 MB/s (2GB/s) have been demonstrated for a 24 disk system with > 40 TB capacity.
Larger capacity HDD based systems are under development.
Pros: Very large capacity, Very high speed, redundancy.
Cons: High cost, large chassis.