2. eXpressWare

Dolphins PCI Express software suite is named eXpressWare and enables applications to communicate over PCI Express cables and backplanes. Several interfaces and APIs are supported, from standard TCP/IP networking to the lowest level direct remote memory access. Each API has its benefits and can be selected based on application requirements. The complete stack can be seen in the figure below:

2.1. SuperSockets

One important part of eXpressWare is SuperSockets which implement a safe and reliable bypass of the TCP/UDP/IP protocol stack for standard socket-based inter-process communication. This bypass moves data directly via the high-performance interconnect and thereby reduces the minimal latency typically by a factor of 10 and more with 100% binary application compatibility, no special configuration and full plug and play.

The SuperSockets software was first released for critical applications in 2004 and has constantly been optimized and maintained to support new versions of Windows and Linux as well as new computer systems and interconnects.

2.2. SISCI

The Dolphin SISCI Developers kit (Software Infrastructure Shared-Memory Cluster Interconnect) consists of driver and API software, tools, documentation and source needed to develop your own embedded application utilizing the low latency and high performance of a PCI Express Cluster. The development kit provides a C system call interface to ease customer integration to Dolphins cluster interconnect. SISCI enables customer application to easily bypass the limitations of traditional network solutions, avoiding time consuming operating system calls, and network protocol software overhead.

The SISCI Software enables applications to directly benefit from remote access time down to 0.54 microseconds and close to wire speed throughput. Direct mapped NTB PIO access gives extremely low transaction overhead and jitter for real-time systems. DMA capabilities provide very high throughput without using CPU for data transfers. Customers can combine PIO and DMA operations for maximum application benefits and performance. The SISCI API also support access to remote IO resources.

eXpressWare 5.5.0 introduces the SmartIO SISCI API extension. This includes functionality enabling SISCI applications to easily access and work with remote PCIe devices.

The SmartIO SISCI API extension is currently only available with Linux. Please contact Dolphin for availability on other operating systems.

More information on the SmartIO SISCI API extension can be found in the updated SISCI Functional specification and at http://dolphinics.com/solutions/SISCI_Smart_IO_PCIe_device_access.html

2.3. Device Lending

Device lending was introduced by eXpressWare 5.5.0 and enables PCI Express devices like NVMes, GPUs, FPGAs and NICs to be accessed over the PCIe fabric without any changes to the device driver software or loss of performance. PCIe TLPs are directly routed between the local server the remote PCIe device. Device Lending is a part of the SmartIO software module.

More information on the Device Lending software can be found at http://www.dolphinics.com/products/pcie_smart_io_device_lending.html and in Chapter 13, SmartIO

2.4. Transparent Hot Add

eXpressWare 5.9.0 introduces support for surprise hot add of local transparent devices. This enables IO devices to be added or removed to running servers. The functionality is made available by the eXpressWare SmartIO module.

More information on Transparent Hot Add can be found at https://www.dolphinics.com/solutions/pcie_hot_add_plug_swap.html and in Chapter 13, SmartIO

2.5. TCP/IP Driver

A TCP/IP network driver was added to the stack in 2011, this enables customers to use PCI Express also for applications that is not compliant with the Berkeley Sockets API, e.g. NFS. The network driver supports standard TCP/IP functionality including ARP and routing to other networks.

The current implementation does not support communication between Linux and Windows systems.

2.6. eXpressWare Licensing

eXpressWare is Developed by Dolphin over more than two decades and consists of both open source and proprietary software. The use of the proprietary software requires a license.

Dolphins NTB enabled PCIe Adapter cards is sold with a license to use the software. Users of eXpressWare on OEM hardware need to obtain the license from the hardware vendor.

A copy of the eXpressWare license text can be found in Appendix E, eXpressWare License text