Guides
Guides

Xtarget Encryption

Suggest Edits

Xtarget data is delivered through the visitor's browser, so encrypting the data is a must. AtData will deliver data via the "eqs" ("encrypted query string") URL parameter, posting to your Xtarget URL.

The "eqs" value is itself a URL query string. If you're adding encryption to an existing account, the eqs parameter will replace your current parameters, and the decrypted eqs value will match what you're currently receiving in the query string portion of your Xtarget destination URL.

Quick start:

  1. Contact AtData's support and provide us with an encryption/decryption key, which works much like a password. The key must be 32 characters in the [0-9, a-f] range, e.g. "d6c60c8e688e0afa7f49416897aadc1f".

📘

Generating a key

The encryption key is not an MD5 hash, but it's the same format. An easy way to generate a key is to come up with a normal password and use any online MD5 tool to generate an MD5 hash of the password.

  1. Update your Xtarget redirect URL to accept and decrypt the "eqs" parameter using our sample code:
    1. Ruby: eqs_decryption_example.rb
    2. Python: eqs_decryption_example.py
    3. Bash: eqs_decryption_example.sh
    4. Perl: eqs_decryption_example.pl
    5. Bash encryption script, which can be used to generate test data: eqs_encryption_example.sh

📘

Decrypted query string

The decrypted value will be in traditional query string format, e.g. "md5_email=337c1f89a68e64e5b8264c8d5121992c&label=this_is_a_data_label", but it will not be URL encoded.


Full Details on AtData's Encrypted Query String

Buckle up, getting encryption/decryption to work across different platforms can be complicated. There are two significant aspects to your ability to decrypt messages produced by our system:

  1. Base64 decoding -- converting the eqs value to a (binary) encrypted string.
  2. Decrypting -- using the aes-128-ecb cipher to decrypt the encrypted string.

Base64 decoding

You must perform Base64 decoding on an encrypted query string prior to decryption. The encrypted string is in binary form, so AtData performs Base64 encoding followed by URL encoding prior to placement in the query string of the destination URL. Our encoding method uses the org.apache.commons.codec.binary.Base64 java class with URL-safe mode enabled. "URL-safe mode" means that the traditional "+" and "/" characters in the Base64 encoded output are converted to "-" and "_". In addition, the CR/LF characters at the end of each 76-character line are converted to spaces prior to URL encoding. The "+" -> "-" conversion is a common Base64 variant and is discussed in RFC 3548, but it not all Base64 implementations recognize this variant (even ones like gnu's base64 utility, which specifically references RFC 3548). The java class references earlier RFC 2045, which does not include this variant. Lastly, traditional Base64 encoding uses the "=" character to pad the end of the encoded string in order to achieve a character count that is divisible by 4. The java class omits this, but some Base64 implementations require it.

Summary of AtData's Base64 variant:

  • "+" (plus) in standard Base64 becomes "-" (dash) in AtData's variant.
  • "/" (forward slash) in standard Bas64 becomes "_" (underscore) in AtData's variant.
  • CR/LF in standard Base64 becomes space+space (URL encoded as "%20%20").
  • Trailing "=" padding is not used in AtData's variant
  • Because of the Base64 variant used by our systems, it's possible that you could face compatibility issues that would affect your ability to use the encryption feature. The items above may need to be reversed for you to decode an encrypted string so it can decrypted. See the sample code in the Quick Start section for additional info.

Base64 equals sign padding

Some base64 decoding methods require trailing equals sign padding to be in place in order to decode a string. This section assumes you have a base64-encoded string with the equals signs removed.

The first step is to remove all whitespace (including "%20", carriage returns, and line feeds) from the encoded string. Calculate the length modulo 4. In simple terms, you'll need to add "=" signs until the length is evenly divisible by 4 before the base64 string can be decoded.

  • If the base64 string has a length mod 4 of 0, don't add any = signs.
  • If the base64 string has a length mod 4 of 2, add two = signs.
  • If the base64 string has a length mod 4 of 3, add one = sign.
  • There will never be a base64 string with a length mod 4 of 1, because base64 uses base-6 and ascii uses base-8, meaning that only three options are available.

Example 1:

  • (Unusable?) base64 string: YWJkb21pbm9hbnRlcmlvcg
  • Length of base64 string: 22
  • Length modulo 4: 2
  • Solution: Two "=" signs need to be added.
  • Usable base64 string: YWJkb21pbm9hbnRlcmlvcg==
  • New length: 24

Example 2:

  • (Unusable?) base64 string: eWVsbG93LXNob3VsZGVyZWQ
  • Length of base64 string: 23
  • Length modulo 4: 1
  • Solution: One "=" sign needs to be added.
  • Usable base64 string: eWVsbG93LXNob3VsZGVyZWQ=
  • New length: 24

Example 3:

  • (Unusable?) base64 string: dHJvdWJsZXNvbWVuZXNz
  • Length of base64 string: 20
  • Length modulo 4: 0
  • Solution: fine as-is
  • Usable base64 string: dHJvdWJsZXNvbWVuZXNz
  • New length: 20

Decryption

AES-128-ECB is an Electronic Codebook (ECB) block cipher, meaning that each block of data is independently encrypted. It is generally thought of as insecure because repeating blocks of input data will produce identical encrypted output. However, this is not an issue for xtarget output, which has no inherent repetition. In addition, it has a small key/block size and does not use an initialization vector, which can ease implementation.

  • Cipher: aes-128-ecb
  • Block size: 16 bytes / 128 bits
  • Key size: 16 bytes / 128 bits
  • Initialization vector: none
  • Padding: PKCS#7

128-bit key

The key is generally represented as a 32-character hex number representing 16 bytes. The key is exchanged in plaintext, but encryption functions generally accept a binary form of the key, where each hex pair is converted to its corresponding binary value.

$ echo d6c60c8e688e0afa7f49416897aadc1f | xxd -r -c 16 -ps > key.bin

Initialization vector (IV)

Some forms of AES encryption use an initialization vector. AES-128-ECB does not. In most implementations, the IV will be ignored once AES-128-ECB is established as the cipher.

PKCS#7 padding

Padding occurs prior to encryption in order to force the pre-encryption plaintext to match the block size of the cipher. Many implentations handle this automatically. Padding is applied in ALL cases, even when the pre-encryption size matches the block size.

PKCS#5 is a subset of PKCS#7 but is limited to an 8-byte block size. Some implentations such as Java incorrectly use PKCS#5 to describe what is actually PKCS#7.

Padding is in whole bytes. The value of each added byte is the number of bytes that are added, i.e. N bytes, each of value N are added. The number of bytes added will depend on the block boundary to which the message needs to be extended.

For a 16-byte block size, the padding added to the pre-encrypted string will be one of:

01 02 02 03 03 03 04 04 04 04 ... 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 0F 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10

Examples of PKCS#7 padding for block size of 16 bytes:

Description              Plaintext data                       Padded plaintext data
 3 bytes (13 pad bytes): FDFDFD                           --> FDFDFD0D0D0D0D0D0D0D0D0D0D0D0D0D
15 bytes ( 1 pad byte ): FDFDFDFDFDFDFDFDFDFDFDFDFDFDFD   --> FDFDFDFDFDFDFDFDFDFDFDFDFDFDFD01
16 bytes (16 pad bytes): FDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFD --> FDFDFDFDFDFDFDFDFDFDFDFDFDFDFDFD10101010101010101010101010101010

Command-line example

Example of base64 standardization and decryption, using encryption key "d6c60c8e688e0afa7f49416897aadc1f":

Raw incoming query string:

eqs=a0dBOP_RYgAncnExb4V-Y6ptSbaSaOq0iDoEsZY6oqJJfJHpTLf_i59couFGlbk5%20%20vK3ZJuSvVV57-no5V5r1C0VMBI0ebVc5_IhJcN9n678

Converted to standard Base64 (also written to example.base64 for the openssl commands below):

a0dBOP/RYgAncnExb4V+Y6ptSbaSaOq0iDoEsZY6oqJJfJHpTLf/i59couFGlbk5vK3ZJuSvVV57+no5V5r1C0VMBI0ebVc5/IhJcN9n678=

Command-line decryption 1 (using openssl only):

$ openssl enc -d -aes-128-ecb -K d6c60c8e688e0afa7f49416897aadc1f -base64 -in example.base64

md5_email=337c1f89a68e64e5b8264c8d5121992c&label=this_is_a_data_label

Command-line decryption 2 (using the base64 utility followed by openssl):

$ cat example.base64 | base64 -d | openssl enc -d -aes-128-ecb -K d6c60c8e688e0afa7f49416897aadc1f

md5_email=337c1f89a68e64e5b8264c8d5121992c&label=this_is_a_data_label

Updated about 1 month ago