SKI Decryption Library

The decryption library is a static C library designed to provide high decryption throughput of TLS records. The underlying packet processing system need not be aware of TLS records, or perform any TLS parsing operations. The underlying packet processing system should perform standard TCP layer termination and re-assembly, and eliminate packet deduplication that could occur if traffic is tapped at multiple points. All packets belonging to a particular TCP session must arrive at the same thread processing the data. Each decryption thread operates as a completely independent system.

../_images/DecryptionLibrary.png

The decryption library supports TLS 1.2, TLS 1.2 PFS, and TLS 1.3 ciphers. The library provides high decryption throughput at 80% of an OpenSSL speed test. The table below shows decryption throughput in Gbps on an AWS m4.xlarge AWS Linux 2, with an Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz, with AES-NI support enabled.

Cipher Block Size Decryption Gbps
AES-128-GCM Mac=AEAD 1024 13.95
AES-256-GCM Mac=AEAD 1024 11.12
CHACHA20-POLY1305 1024 5.86
AES-128-CBC-HMAC=SHA1 1024 4.01
AES-128-CBC-HMAC=SHA256 1024 2.04
AES-256-CBC-HMAC-SHA1 1024 3.80
AES-256-CBC-HMAC-SHA256 1024 1.97

Note

The numbers above are for a single thread. Higher throughput can be achieved by running multiple decryption threads.

Using the Library

../_images/DecryptLibHighlighted.png

Data Types

  • Thread Info (nuthread) Some per thread info is required in order to keep configuration information and re-usable buffers. This per thread information is represented by the nuthread structure type. It is expected that this information be consistent for every function call in the same thread.
  • Session Info (nusession) Session state information is required so that the same thread can handle multiple independent sessions. The session info pointer is expected to be saved in the same lookup table as the TCP session information and passed into the decryptor so as to have the decryptor be up to date with current TLS record processing for this session. The contents of this pointer is opaque to the user of the library.
  • Packet Info (nupacket) Each record that is passed into the library also requires information to determine directionality (was it sent from the client to the server or the server to the client). This packet information is represented by the nupacket structure.

Library Functions

The decryption library interface has the following functions:

nu_record_processor_init

This is the initialization function.

  • Parameters:
    nu_processor_options: a structure containing two function pointers: a function to lookup master keys based on a client-random value, and a function to free key memory. nu_processor_init_response: An output parameter which indicates a successful initialization, or that invalid data was passed to the function. The latter is likely due to null pointers.
  • Returns:
    nutldh: Nubeva thread local data handle.

nu_new_session

Called at the beginning of a new session in order to initialize the session state information.

  • Inputs:
    nutldh: The nubeva thread local data handle that is returned from the call to nu_record processor_init().
  • Returns:
    nush: a new nubeva session handle.

nu_record_processor

This is the actual record processor. It will keep state information and decrypt the data and return the decrypted data and status.

  • Parameters:

    nush: The nubeva session handle returned from the call to nu_new_session().

    nupacket: The input packet information.

    indata: A buffer containing the encrypted data (TCP payload).

    outdata: a buffer to contain the decrypted data. The size of this buffer should be the maximal size of a TLS record 16384 + 1024 (17408) + len (the size of the input data buffer).

    len: On input len indicates the length of the input data. On output len indicates the length of the decrypted data in the output buffer.

  • Returns:

    record processing status enumeration.

nu_record_processor_cleanup

This is the cleanup function. It releases memory and closes all resources used.

  • Parameters:
    nutldh: the nubeva thread local data handle.

nu_session_cleanup

Called when freeing a TLS session to release all session records.

  • Parameters:
    nush: the nubeva session handle.

nu_lost_client_frame

Should be called to notify the library of the first loss in a consecutive set of lost packets from the client to the server.

  • Parameters:
    nush: the nubeva session handle.

nu_lost_server_frame

Should be called to notify the library of the first loss in a consecutive set of lost packets from the server to the client.

  • Parameters:
    nush: the nubeva session handle.

nu_license_check

  • Returns:
    A string containing the library license type and expiration date.

Initialization

Call nu_record_processor_init() with configuration parameters. Check for errors. After the initialization, you will have a nuthread structure instance that you should pass into every process request.

Processing TLS Records

For each new TCP Session: Call nu_new_session() to get a session state pointer. Keep track of this pointer within your TCP state information structure.

For each TCP packet generate a nupacket info structure with information about this particular packet. Pass the nusession and nupacket structs, together with an encrypted packet, a buffer for decrypted TLS data, and a pointer to length information to nu_record_processor(). The return code indicates what type of data has been decrypted or if any errors occurred.

The following section explains how nu_record_processor() parses and decrypts all possible TLS encapsulations in TCP payloads:

  • If the TCP payload contains a single complete TLS record (shown below):

    ../_images/TCPTLS001.png

    nu_record_processor() outputs the decrypted data in the outdata buffer, and len is set to the length of the decrypted data. The return value indicates the type of TLS record that was decrypted: RECORD_HANDSHAKE_DECRYPTED, RECORD_DECRYPTED, or RECORD_ALERT_DECRYPTED.

  • If the TCP payload contains multiple TLS records (shown below):

    ../_images/TCPTLS002.png

    nu_record_processor() outputs the decrypted data from all the TLS records in the outdata buffer, and len is set to the length of the decrypted data. The return value is RECORD_HANDSHAKE_DECRYPTED, RECORD_DECRYPTED, or RECORD_ALERT_DECRYPTED.

  • If the TCP payload contains one or more TLS records followed by the first fragment of the next TLS record (shown below):

    ../_images/TCPTLS003.png

    nu_record_processor() outputs the decrypted data from all the TLS records in the outdata buffer, and len is set to the length of the decrypted data. The return value is RECORD_NO_ERROR, indicating that more packet payload inputs are required to complete the next TLS record.

  • If the TCP payload contains a fragment of a TLS record which is not the last fragment (shown below):

    ../_images/TCPTLS004.png

    nu_record_processor() sets the value of len to 0 and returns RECORD_NO_ERROR.

  • If the TCP payload contains the last fragment of a TLS record and one or more TLS records (shown below):

    ../_images/TCPTLS005.png

    nu_record_processor() outputs the decrypted data from all the TLS records in the outdata buffer, and len is set to the length of the decrypted data. The return value is RECORD_HANDSHAKE_DECRYPTED, RECORD_DECRYPTED, or RECORD_ALERT_DECRYPTED.

Note

A return value other than RECORD_HANDSHAKE_DECRYPTED, RECORD_DECRYPTED, RECORD_ALERT_DECRYPTED or RECORD_NO_ERROR is an indication that the library will not continue decrypting the TLS session. The caller can drop the session state returned from nu_new_session().

Recovering from Lost Packets

The library attempts to recover lost packets for AES_128_GCM and AES_256_GCM. You need to notify the library Directionality matters. Your TCP processing logic should notify the library of the first loss in a consecutive set of lost packets. Since only complete TLS records can be decrypted, the library will wait for the next TLS record header after a loss is reported when attempting to recover from lost packets. Call nu_lost_client_frame() if frames from the client to the server were lost. Call nu_lost_server_frame() of frames from the server to the client were lost.

Cleanup

Before cleaning up a TCP session, you should ensure that nu_session_cleanup() is called in order to release session memory. Before stopping a thread, you should ensure that nu_record_processor_cleanup() is called in order to release memory.

Sample Code and Test Files

Users who have licensed the decryption library have access to a repository containing:

  • The decryption library binary file.
  • A sample test program that reads in PCAP files, passes TCP payloads to the library, which returns decrypted TLS records.
  • Documented header files containing the function declarations and data structure definitions required for library initialization, session initialization, processing TLS records, notifying of lost packets, cleanup and license display.
  • A Makefile to compile the program.
  • A directory of test cases.
  • A Readme file with instruction how to run performance and decryption validation tests.
  • A file containing frequently asked questions.

The sample program shows how to pass TCP payloads to the decryption library. TCP records without a payload (SYN, ACK, SYN/ACK, RST, FIN) need not be passed to the library. Packets should be passed in order (see specifics in the FAQ section).

Note

A TCP packet could be quite large in environments where NICs coalesce packets to reduce the number of packets forwarded up the stack. This could be the case when the decryption library is used in out-of-band cases where the MTU can be large.

Frequently Asked Questions

Note

A more detailed set of questions is available in the repositories provided to organizations that have licensed Nubeva solutions.

Q: Is a complete TCP session required for decryption?
The decryption library does not need to receive TCP packets that do not contain a payload such as SYN, SYN/ACK, plain ACK, RST, FIN. All other packets should be passed to the decryption library.
Q: Is it possible to decrypt application records without decrypting the handshake?
Handshake information must be available. The encryption/decryption/MAC starts at the end of the TLS 1.2 handshake and in TLS 1.3, the handshake is mostly encrypted. The handshake information is required in order to decrypt the application data that follows the handshake.
Q: Does nu_record_processor() need TCP packets in the exact order?
Packets containing TLS handshakes must be passed in the right order. For TLS 1.2 the handshake sequence is:
  • Client → server: client hello
  • Server → client: server hello, server certificate, server key exchange, server hello done
  • Client → server: client key exchange, change cipher spec, client handshake finished
  • Server → client: server change cipher spec, server handshake finished.

For TLS 1.3 the sequence is:

  • Client → server: client hello
  • Server → client: server hello, change cipher spec, certificate, handshake finished
  • Client → server: change cipher spec, handshake finished

The caller does not need to parse these records. Once the handshake is complete the order needs to be maintained in each direction.

Q: What should the size of the decryption buffer be?
The decrypted buffer length must be at least 17408 (max TLS record length) + size of the input length.
Q: How does the decryption library match a TLS session with a master key?
There is a callback function pointer that is set by the caller when the library is initialized. The library calls this function with a client-random value contained in client-hello messages.
Q: Do packets need to be buffered until keys are available?
You should not have to buffer packets when using Nubeva Sensors to receive keys. Nubeva Sensors send keys within 200usec of key creation. Therefore keys get to the destination well before anything needs to be decrypted.
Q: What happens if the key lookup function fails to find a key?
The nu_record_processor will return as error status indicating that a key was not found.
Q: Does the library recover from lost packets?
The library currently attempts to recover lost packets for AES_128_GCM and AES_256_GCM provided that not more than 128 packets were lost. There is a function to notify the library of lost packets from a client to the server and a function to notify the library of lost packets from the server to the client.
Q: Where can I find detailed information about TLS records?
These are two good references: TLS 1.2 and TLS 1.3.