Manual Page Search Parameters
RSA_PUBLIC_ENCRYPT(3) Library Functions Manual RSA_PUBLIC_ENCRYPT(3)

RSA_public_encrypt, RSA_private_decrypt, EVP_PKEY_encrypt_old, EVP_PKEY_decrypt_old — RSA public key cryptography

#include <openssl/rsa.h>

int
RSA_public_encrypt(int flen, const unsigned char *from, unsigned char *to, RSA *rsa, int padding);

int
RSA_private_decrypt(int flen, const unsigned char *from, unsigned char *to, RSA *rsa, int padding);

#include <openssl/evp.h>

int
EVP_PKEY_encrypt_old(unsigned char *to, const unsigned char *from, int flen, EVP_PKEY *pkey);

int
EVP_PKEY_decrypt_old(unsigned char *to, const unsigned char *from, int flen, EVP_PKEY *pkey);

() encrypts the flen bytes at from (usually a session key) using the public key rsa and stores the ciphertext in to. to must point to RSA_size(rsa) bytes of memory.

padding denotes one of the following modes:

PKCS #1 v1.5 padding. This currently is the most widely used mode.
EME-OAEP as defined in PKCS #1 v2.0 with SHA-1, MGF1 and an empty encoding parameter. This mode is recommended for all new applications.
Raw RSA encryption. This mode should only be used to implement cryptographically sound padding modes in the application code. Encrypting user data directly with RSA is insecure.

flen must be less than (rsa) - 11 for the PKCS #1 v1.5 based padding modes, less than RSA_size(rsa) - 41 for RSA_PKCS1_OAEP_PADDING and exactly RSA_size(rsa) for RSA_NO_PADDING.

() decrypts the flen bytes at from using the private key rsa and stores the plaintext in to. to must point to a memory section large enough to hold the decrypted data (which is smaller than RSA_size(rsa)). padding is the padding mode that was used to encrypt the data.

() is a deprecated wrapper around RSA_public_encrypt() that uses the RSA public key stored in pkey and RSA_PKCS1_PADDING.

() is a deprecated wrapper around RSA_private_decrypt() that uses the RSA private key stored in pkey and RSA_PKCS1_PADDING.

RSA_public_encrypt() and EVP_PKEY_encrypt_old() return the size of the encrypted data (i.e. RSA_size(rsa)). RSA_private_decrypt() and EVP_PKEY_decrypt_old() returns the size of the recovered plaintext. On error, -1 is returned; the error codes can be obtained by ERR_get_error(3).

In addition to the return values documented above, EVP_PKEY_encrypt_old() may return 0 if the EVP_PKEY_id(3) of pkey is not EVP_PKEY_RSA.

EVP_PKEY_decrypt(3), EVP_PKEY_encrypt(3), RSA_meth_set_priv_dec(3), RSA_new(3), RSA_size(3)

SSL, PKCS #1 v2.0

RSA_public_encrypt() and RSA_private_decrypt() appeared in SSLeay 0.4 or earlier and have been available since OpenBSD 2.4.

EVP_PKEY_encrypt() and EVP_PKEY_decrypt() first appeared in SSLeay 0.9.0 and have been available since OpenBSD 2.4. There were renamed to EVP_PKEY_encrypt_old() and EVP_PKEY_decrypt_old() in OpenSSL 1.0.0 and OpenBSD 4.9.

RSA_NO_PADDING is available since SSLeay 0.9.0. OAEP was added in OpenSSL 0.9.2b.

Decryption failures in the RSA_PKCS1_PADDING mode leak information which can potentially be used to mount a Bleichenbacher padding oracle attack. This is an inherent weakness in the PKCS #1 v1.5 padding design. Prefer RSA_PKCS1_OAEP_PADDING.

September 10, 2023 openbsd