Edit File: ssleay.txt
Bundle of old SSLeay documentation files [OBSOLETE!] *** WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! *** OBSOLETE means that nothing in this document should be trusted. This document is provided mostly for historical purposes (it wasn't even up to date at the time SSLeay 0.8.1 was released) and as inspiration. If you copy some snippet of code from this document, please _check_ that it really is correct from all points of view. For example, you can check with the other documents in this directory tree, or by comparing with relevant parts of the include files. People have done the mistake of trusting what's written here. Please don't do that. *** WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! *** ==== readme ======================================================== This is the old 0.6.6 docuementation. Most of the cipher stuff is still relevent but I'm working (very slowly) on new documentation. The current version can be found online at http://www.cryptsoft.com/ssleay/doc ==== API.doc ======================================================== SSL - SSLv2/v3/v23 etc. BIO - methods and how they plug together MEM - memory allocation callback CRYPTO - locking for threads EVP - Ciphers/Digests/signatures RSA - methods X509 - certificate retrieval X509 - validation X509 - X509v3 extensions Objects - adding object identifiers ASN.1 - parsing PEM - parsing ==== ssl/readme ===================================================== 22 Jun 1996 This file belongs in ../apps, but I'll leave it here because it deals with SSL :-) It is rather dated but it gives you an idea of how things work. === 17 Jul 1995 I have been changing things quite a bit and have not fully updated this file, so take what you read with a grain of salt eric === The s_client and s_server programs can be used to test SSL capable IP/port addresses and the verification of the X509 certificates in use by these services. I strongly advise having a look at the code to get an idea of how to use the authentication under SSLeay. Any feedback on changes and improvements would be greatly accepted. This file will probably be gibberish unless you have read rfc1421, rfc1422, rfc1423 and rfc1424 which describe PEM authentication. A Brief outline (and examples) how to use them to do so. NOTE: The environment variable SSL_CIPER is used to specify the prefered cipher to use, play around with setting it's value to combinations of RC4-MD5, EXP-RC4-MD5, CBC-DES-MD5, CBC3-DES-MD5, CFB-DES-NULL in a : separated list. This directory contains 3 X509 certificates which can be used by these programs. client.pem: a file containing a certificate and private key to be used by s_client. server.pem :a file containing a certificate and private key to be used by s_server. eay1024.pem:the certificate used to sign client.pem and server.pem. This would be your CA's certificate. There is also a link from the file a8556381.0 to eay1024.PEM. The value a8556381 is returned by 'x509 -hash -noout <eay1024.pem' and is the value used by X509 verification routines to 'find' this certificte when search a directory for it. [the above is not true any more, the CA cert is ../certs/testca.pem which is signed by ../certs/mincomca.pem] When testing the s_server, you may get bind: Address already in use errors. These indicate the port is still being held by the unix kernel and you are going to have to wait for it to let go of it. If this is the case, remember to use the port commands on the s_server and s_client to talk on an alternative port. ===== s_client. This program can be used to connect to any IP/hostname:port that is talking SSL. Once connected, it will attempt to authenticate the certificate it was passed and if everything works as expected, a 2 directional channel will be open. Any text typed will be sent to the other end. type Q<cr> to exit. Flags are as follows. -host arg : Arg is the host or IP address to connect to. -port arg : Arg is the port to connect to (https is 443). -verify arg : Turn on authentication of the server certificate. : Arg specifies the 'depth', this will covered below. -cert arg : The optional certificate to use. This certificate : will be returned to the server if the server : requests it for client authentication. -key arg : The private key that matches the certificate : specified by the -cert option. If this is not : specified (but -cert is), the -cert file will be : searched for the Private key. Both files are : assumed to be in PEM format. -CApath arg : When to look for certificates when 'verifying' the : certificate from the server. -CAfile arg : A file containing certificates to be used for : 'verifying' the server certificate. -reconnect : Once a connection has been made, drop it and : reconnect with same session-id. This is for testing :-). The '-verify n' parameter specifies not only to verify the servers certificate but to also only take notice of 'n' levels. The best way to explain is to show via examples. Given s_server -cert server.PEM is running. s_client CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo server issuer= /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify error:num=1:unable to get issuer certificate verify return:1 CIPHER is CBC-DES-MD5 What has happened is that the 'SSLeay demo server' certificate's issuer ('CA') could not be found but because verify is not on, we don't care and the connection has been made anyway. It is now 'up' using CBC-DES-MD5 mode. This is an unauthenticate secure channel. You may not be talking to the right person but the data going to them is encrypted. s_client -verify 0 CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo server issuer= /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify error:num=1:unable to get issuer certificate verify return:1 CIPHER is CBC-DES-MD5 We are 'verifying' but only to depth 0, so since the 'SSLeay demo server' certificate passed the date and checksum, we are happy to proceed. s_client -verify 1 CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo server issuer= /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify error:num=1:unable to get issuer certificate verify return:0 ERROR verify error:unable to get issuer certificate In this case we failed to make the connection because we could not authenticate the certificate because we could not find the 'CA' certificate. s_client -verify 1 -CAfile eay1024.PEM CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo server verify return:1 depth=1 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify return:1 CIPHER is CBC-DES-MD5 We loaded the certificates from the file eay1024.PEM. Everything checked out and so we made the connection. s_client -verify 1 -CApath . CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo server verify return:1 depth=1 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify return:1 CIPHER is CBC-DES-MD5 We looked in out local directory for issuer certificates and 'found' a8556381.0 and so everything is ok. It is worth noting that 'CA' is a self certified certificate. If you are passed one of these, it will fail to 'verify' at depth 0 because we need to lookup the certifier of a certificate from some information that we trust and keep locally. SSL_CIPHER=CBC3-DES-MD5:RC4-MD5 export SSL_CIPHER s_client -verify 10 -CApath . -reconnect CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo server verify return:1 depth=1 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify return:1 drop the connection and reconnect with the same session id CIPHER is CBC3-DES-MD5 This has done a full connection and then re-estabished it with the same session id but a new socket. No RSA stuff occures on the second connection. Note that we said we would prefer to use CBC3-DES-MD5 encryption and so, since the server supports it, we are. ===== s_server This program accepts SSL connections on a specified port Once connected, it will estabish an SSL connection and optionaly attempt to authenticate the client. A 2 directional channel will be open. Any text typed will be sent to the other end. Type Q<cr> to exit. Flags are as follows. -port arg : Arg is the port to listen on. -verify arg : Turn on authentication of the client if they have a : certificate. Arg specifies the 'depth'. -Verify arg : Turn on authentication of the client. If they don't : have a valid certificate, drop the connection. -cert arg : The certificate to use. This certificate : will be passed to the client. If it is not : specified, it will default to server.PEM -key arg : The private key that matches the certificate : specified by the -cert option. If this is not : specified (but -cert is), the -cert file will be : searched for the Private key. Both files are : assumed to be in PEM format. Default is server.PEM -CApath arg : When to look for certificates when 'verifying' the : certificate from the client. -CAfile arg : A file containing certificates to be used for : 'verifying' the client certificate. For the following 'demo' I will specify the s_server command and the s_client command and then list the output from the s_server. s_server s_client CONNECTED CIPHER is CBC-DES-MD5 Everything up and running s_server -verify 0 s_client CONNECTED CIPHER is CBC-DES-MD5 Ok since no certificate was returned and we don't care. s_server -verify 0 ./s_client -cert client.PEM CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo client issuer= /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify error:num=1:unable to get issuer certificate verify return:1 CIPHER is CBC-DES-MD5 Ok since we were only verifying to level 0 s_server -verify 4 s_client -cert client.PEM CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo client issuer= /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify error:num=1:unable to get issuer certificate verify return:0 ERROR verify error:unable to get issuer certificate Bad because we could not authenticate the returned certificate. s_server -verify 4 -CApath . s_client -cert client.PEM CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo client verify return:1 depth=1 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify return:1 CIPHER is CBC-DES-MD5 Ok because we could authenticate the returned certificate :-). s_server -Verify 0 -CApath . s_client CONNECTED ERROR SSL error:function is:REQUEST_CERTIFICATE :error is :client end did not return a certificate Error because no certificate returned. s_server -Verify 4 -CApath . s_client -cert client.PEM CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo client verify return:1 depth=1 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify return:1 CIPHER is CBC-DES-MD5 Full authentication of the client. So in summary to do full authentication of both ends s_server -Verify 9 -CApath . s_client -cert client.PEM -CApath . -verify 9 From the server side CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo client verify return:1 depth=1 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify return:1 CIPHER is CBC-DES-MD5 From the client side CONNECTED depth=0 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=SSLeay demo server verify return:1 depth=1 /C=AU/SOP=QLD/O=Mincom Pty. Ltd./OU=CS/CN=CA verify return:1 CIPHER is CBC-DES-MD5 For general probing of the 'internet https' servers for the distribution area, run s_client -host www.netscape.com -port 443 -verify 4 -CApath ../rsa/hash Then enter GET / and you should be talking to the https server on that host. www.rsa.com was refusing to respond to connections on 443 when I was testing. have fun :-). eric ==== a_verify.doc ======================================================== From eay@mincom.com Fri Oct 4 18:29:06 1996 Received: by orb.mincom.oz.au id AA29080 (5.65c/IDA-1.4.4 for eay); Fri, 4 Oct 1996 08:29:07 +1000 Date: Fri, 4 Oct 1996 08:29:06 +1000 (EST) From: Eric Young <eay@mincom.oz.au> X-Sender: eay@orb To: wplatzer <wplatzer@iaik.tu-graz.ac.at> Cc: Eric Young <eay@mincom.oz.au>, SSL Mailing List <ssl-users@mincom.com> Subject: Re: Netscape's Public Key In-Reply-To: <19961003134837.NTM0049@iaik.tu-graz.ac.at> Message-Id: <Pine.SOL.3.91.961004081346.8018K-100000@orb> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO X-Status: On Thu, 3 Oct 1996, wplatzer wrote: > I get Public Key from Netscape (Gold 3.0b4), but cannot do anything > with it... It looks like (asn1parse): > > 0:d=0 hl=3 l=180 cons: SEQUENCE > 3:d=1 hl=2 l= 96 cons: SEQUENCE > 5:d=2 hl=2 l= 92 cons: SEQUENCE > 7:d=3 hl=2 l= 13 cons: SEQUENCE > 9:d=4 hl=2 l= 9 prim: OBJECT :rsaEncryption > 20:d=4 hl=2 l= 0 prim: NULL > 22:d=3 hl=2 l= 75 prim: BIT STRING > 99:d=2 hl=2 l= 0 prim: IA5STRING : > 101:d=1 hl=2 l= 13 cons: SEQUENCE > 103:d=2 hl=2 l= 9 prim: OBJECT :md5withRSAEncryption > 114:d=2 hl=2 l= 0 prim: NULL > 116:d=1 hl=2 l= 65 prim: BIT STRING > > The first BIT STRING is the public key and the second BIT STRING is > the signature. > But a public key consists of the public exponent and the modulus. Are > both numbers in the first BIT STRING? > Is there a document simply describing this coding stuff (checking > signature, get the public key, etc.)? Minimal in SSLeay. If you want to see what the modulus and exponent are, try asn1parse -offset 25 -length 75 <key.pem asn1parse will currently stuff up on the 'length 75' part (fixed in next release) but it will print the stuff. If you are after more documentation on ASN.1, have a look at www.rsa.com and get their PKCS documents, most of my initial work on SSLeay was done using them. As for SSLeay, util/crypto.num and util/ssl.num are lists of all exported functions in the library (but not macros :-(. The ones for extracting public keys from certificates and certificate requests are EVP_PKEY * X509_REQ_extract_key(X509_REQ *req); EVP_PKEY * X509_extract_key(X509 *x509); To verify a signature on a signed ASN.1 object int X509_verify(X509 *a,EVP_PKEY *key); int X509_REQ_verify(X509_REQ *a,EVP_PKEY *key); int X509_CRL_verify(X509_CRL *a,EVP_PKEY *key); int NETSCAPE_SPKI_verify(NETSCAPE_SPKI *a,EVP_PKEY *key); I should mention that EVP_PKEY can be used to hold a public or a private key, since for things like RSA and DSS, a public key is just a subset of what is stored for the private key. To sign any of the above structures int X509_sign(X509 *a,EVP_PKEY *key,EVP_MD *md); int X509_REQ_sign(X509_REQ *a,EVP_PKEY *key,EVP_MD *md); int X509_CRL_sign(X509_CRL *a,EVP_PKEY *key,EVP_MD *md); int NETSCAPE_SPKI_sign(NETSCAPE_SPKI *a,EVP_PKEY *key,EVP_MD *md); where md is the message digest to sign with. There are all defined in x509.h and all the _sign and _verify functions are actually macros to the ASN1_sign() and ASN1_verify() functions. These functions will put the correct algorithm identifiers in the correct places in the structures. eric -- Eric Young | BOOL is tri-state according to Bill Gates. AARNet: eay@mincom.oz.au | RTFM Win32 GetMessage(). ==== x509 ======================================================= X509_verify() X509_sign() X509_get_version() X509_get_serialNumber() X509_get_issuer() X509_get_subject() X509_get_notBefore() X509_get_notAfter() X509_get_pubkey() X509_set_version() X509_set_serialNumber() X509_set_issuer() X509_set_subject() X509_set_notBefore() X509_set_notAfter() X509_set_pubkey() X509_get_extensions() X509_set_extensions() X509_EXTENSIONS_clear() X509_EXTENSIONS_retrieve() X509_EXTENSIONS_add() X509_EXTENSIONS_delete() ==== x509 attribute ================================================ PKCS7 STACK of X509_ATTRIBUTES ASN1_OBJECT STACK of ASN1_TYPE So it is p7.xa[].obj p7.xa[].data[] get_obj_by_nid(STACK , nid) get_num_by_nid(STACK , nid) get_data_by_nid(STACK , nid, index) X509_ATTRIBUTE *X509_ATTRIBUTE_new(void ); void X509_ATTRIBUTE_free(X509_ATTRIBUTE *a); X509_ATTRIBUTE *X509_ATTRIBUTE_create_by_NID(X509_ATTRIBUTE **ex, int nid, STACK *value); X509_ATTRIBUTE *X509_ATTRIBUTE_create_by_OBJ(X509_ATTRIBUTE **ex, int nid, STACK *value); int X509_ATTRIBUTE_set_object(X509_ATTRIBUTE *ex,ASN1_OBJECT *obj); int X509_ATTRIBUTE_add_data(X509_ATTRIBUTE *ex, int index, ASN1_TYPE *value); ASN1_OBJECT * X509_ATTRIBUTE_get_object(X509_ATTRIBUTE *ex); int X509_ATTRIBUTE_get_num(X509_ATTRIBUTE *ne); ASN1_TYPE * X509_ATTRIBUTE_get_data(X509_ATTRIBUTE *ne,int index); ASN1_TYPE * X509_ATTRIBUTE_get_data_by_NID(X509_ATTRIBUTE *ne, ASN1_OBJECT *obj); X509_ATTRIBUTE *PKCS7_get_s_att_by_NID(PKCS7 *p7,int nid); X509_ATTRIBUTE *PKCS7_get_u_att_by_NID(PKCS7 *p7,int nid); ==== x509 v3 ======================================================== The 'new' system. The X509_EXTENSION_METHOD includes extensions and attributes and/or names. Basically everthing that can be added to an X509 with an OID identifying it. It operates via 2 methods per object id. int a2i_XXX(X509 *x,char *str,int len); int i2a_XXX(BIO *bp,X509 *x); The a2i_XXX function will add the object with a value converted from the string into the X509. Len can be -1 in which case the length is calculated via strlen(str). Applications can always use direct knowledge to load and unload the relevent objects themselves. i2a_XXX will print to the passed BIO, a text representation of the relevet object. Use a memory BIO if you want it printed to a buffer :-). X509_add_by_NID(X509 *x,int nid,char *str,int len); X509_add_by_OBJ(X509 *x,ASN1_OBJECT *obj,char *str,int len); X509_print_by_name(BIO *bp,X509 *x); X509_print_by_NID(BIO *bp,X509 *x); X509_print_by_OBJ(BIO *bp,X509 *x); ==== verify ======================================================== X509_verify_cert_chain( CERT_STORE *cert_store, STACK /* X509 */ *certs, int *verify_result, int (*verify_error_callback)() char *argument_to_callback, /* SSL */ app_verify_callback( char *app_verify_arg, /* from SSL_CTX */ STACK /* X509 */ *certs, int *verify_result, int (*verify_error_callback)() SSL *s, int X509_verify_cert( CERT_STORE *cert_store, X509 *x509, int *verify_result, int (*verify_error_callback)(), char *arg, ==== apps.doc ======================================================== The applications Ok, where to begin.... In the begining, when SSLeay was small (April 1995), there were but few applications, they did happily cohabit in the one bin directory. Then over time, they did multiply and grow, and they started to look like microsoft software; 500k to print 'hello world'. A new approach was needed. They were coalessed into one 'Monolithic' application, ssleay. This one program is composed of many programs that can all be compiled independantly. ssleay has 3 modes of operation. 1) If the ssleay binary has the name of one of its component programs, it executes that program and then exits. This can be achieved by using hard or symbolic links, or failing that, just renaming the binary. 2) If the first argument to ssleay is the name of one of the component programs, that program runs that program and then exits. 3) If there are no arguments, ssleay enters a 'command' mode. Each line is interpreted as a program name plus arguments. After each 'program' is run, ssleay returns to the comand line. dgst - message digests enc - encryption and base64 encoding ans1parse - 'pulls' appart ASN.1 encoded objects like certificates. dh - Diffle-Hellman parameter manipulation. rsa - RSA manipulations. crl - Certificate revokion list manipulations x509 - X509 cert fiddles, including signing. pkcs7 - pkcs7 manipulation, only DER versions right now. genrsa - generate an RSA private key. gendh - Generate a set of Diffle-Hellman parameters. req - Generate a PKCS#10 object, a certificate request. s_client - SSL client program s_server - SSL server program s_time - A SSL protocol timing program s_mult - Another SSL server, but it multiplexes connections. s_filter - under development errstr - Convert SSLeay error numbers to strings. ca - Sign certificate requests, and generate certificate revokion lists crl2pkcs7 - put a crl and certifcates into a pkcs7 object. speed - Benchmark the ciphers. verify - Check certificates hashdir - under development [ there a now a few more options, play with the program to see what they are ] ==== asn1.doc ======================================================== The ASN.1 Routines. ASN.1 is a specification for how to encode structured 'data' in binary form. The approach I have take to the manipulation of structures and their encoding into ASN.1 is as follows. For each distinct structure there are 4 function of the following form TYPE *TYPE_new(void); void TYPE_free(TYPE *); TYPE *d2i_TYPE(TYPE **a,unsigned char **pp,long length); long i2d_TYPE(TYPE *a,unsigned char **pp); /* CHECK RETURN VALUE */ where TYPE is the type of the 'object'. The TYPE that have these functions can be in one of 2 forms, either the internal C malloc()ed data structure or in the DER (a variant of ASN.1 encoding) binary encoding which is just an array of unsigned bytes. The 'i2d' functions converts from the internal form to the DER form and the 'd2i' functions convert from the DER form to the internal form. The 'new' function returns a malloc()ed version of the structure with all substructures either created or left as NULL pointers. For 'optional' fields, they are normally left as NULL to indicate no value. For variable size sub structures (often 'SET OF' or 'SEQUENCE OF' in ASN.1 syntax) the STACK data type is used to hold the values. Have a read of stack.doc and have a look at the relevant header files to see what I mean. If there is an error while malloc()ing the structure, NULL is returned. The 'free' function will free() all the sub components of a particular structure. If any of those sub components have been 'removed', replace them with NULL pointers, the 'free' functions are tolerant of NULL fields. The 'd2i' function copies a binary representation into a C structure. It operates as follows. 'a' is a pointer to a pointer to the structure to populate, 'pp' is a pointer to a pointer to where the DER byte string is located and 'length' is the length of the '*pp' data. If there are no errors, a pointer to the populated structure is returned. If there is an error, NULL is returned. Errors can occur because of malloc() failures but normally they will be due to syntax errors in the DER encoded data being parsed. It is also an error if there was an attempt to read more that 'length' bytes from '*p'. If everything works correctly, the value in '*p' is updated to point at the location just beyond where the DER structure was read from. In this way, chained calls to 'd2i' type functions can be made, with the pointer into the 'data' array being 'walked' along the input byte array. Depending on the value passed for 'a', different things will be done. If 'a' is NULL, a new structure will be malloc()ed and returned. If '*a' is NULL, a new structure will be malloc()ed and put into '*a' and returned. If '*a' is not NULL, the structure in '*a' will be populated, or in the case of an error, free()ed and then returned. Having these semantics means that a structure can call a 'd2i' function to populate a field and if the field is currently NULL, the structure will be created. The 'i2d' function type is used to copy a C structure to a byte array. The parameter 'a' is the structure to convert and '*p' is where to put it. As for the 'd2i' type structure, 'p' is updated to point after the last byte written. If p is NULL, no data is written. The function also returns the number of bytes written. Where this becomes useful is that if the function is called with a NULL 'p' value, the length is returned. This can then be used to malloc() an array of bytes and then the same function can be recalled passing the malloced array to be written to. e.g. int len; unsigned char *bytes,*p; len=i2d_X509(x,NULL); /* get the size of the ASN1 encoding of 'x' */ if ((bytes=(unsigned char *)malloc(len)) == NULL) goto err; p=bytes; i2d_X509(x,&p); Please note that a new variable, 'p' was passed to i2d_X509. After the call to i2d_X509 p has been incremented by len bytes. Now the reason for this functional organisation is that it allows nested structures to be built up by calling these functions as required. There are various macros used to help write the general 'i2d', 'd2i', 'new' and 'free' functions. They are discussed in another file and would only be used by some-one wanting to add new structures to the library. As you might be able to guess, the process of writing ASN.1 files can be a bit CPU expensive for complex structures. I'm willing to live with this since the simpler library code make my life easier and hopefully most programs using these routines will have their execution profiles dominated by cipher or message digest routines. What follows is a list of 'TYPE' values and the corresponding ASN.1 structure and where it is used. TYPE ASN.1 ASN1_INTEGER INTEGER ASN1_BIT_STRING BIT STRING ASN1_OCTET_STRING OCTET STRING ASN1_OBJECT OBJECT IDENTIFIER ASN1_PRINTABLESTRING PrintableString ASN1_T61STRING T61String ASN1_IA5STRING IA5String ASN1_UTCTIME UTCTime ASN1_TYPE Any of the above mentioned types plus SEQUENCE and SET Most of the above mentioned types are actualled stored in the ASN1_BIT_STRING type and macros are used to differentiate between them. The 3 types used are typedef struct asn1_object_st { /* both null if a dynamic ASN1_OBJECT, one is * defined if a 'static' ASN1_OBJECT */ char *sn,*ln; int nid; int length; unsigned char *data; } ASN1_OBJECT; This is used to store ASN1 OBJECTS. Read 'objects.doc' for details ono routines to manipulate this structure. 'sn' and 'ln' are used to hold text strings that represent the object (short name and long or lower case name). These are used by the 'OBJ' library. 'nid' is a number used by the OBJ library to uniquely identify objects. The ASN1 routines will populate the 'length' and 'data' fields which will contain the bit string representing the object. typedef struct asn1_bit_string_st { int length; int type; unsigned char *data; } ASN1_BIT_STRING; This structure is used to hold all the other base ASN1 types except for ASN1_UTCTIME (which is really just a 'char *'). Length is the number of bytes held in data and type is the ASN1 type of the object (there is a list in asn1.h). typedef struct asn1_type_st { int type; union { char *ptr; ASN1_INTEGER * integer; ASN1_BIT_STRING * bit_string; ASN1_OCTET_STRING * octet_string; ASN1_OBJECT * object; ASN1_PRINTABLESTRING * printablestring; ASN1_T61STRING * t61string; ASN1_IA5STRING * ia5string; ASN1_UTCTIME * utctime; ASN1_BIT_STRING * set; ASN1_BIT_STRING * sequence; } value; } ASN1_TYPE; This structure is used in a few places when 'any' type of object can be expected. X509 Certificate X509_CINF CertificateInfo X509_ALGOR AlgorithmIdentifier X509_NAME Name X509_NAME_ENTRY A single sub component of the name. X509_VAL Validity X509_PUBKEY SubjectPublicKeyInfo The above mentioned types are declared in x509.h. They are all quite straight forward except for the X509_NAME/X509_NAME_ENTRY pair. A X509_NAME is a STACK (see stack.doc) of X509_NAME_ENTRY's. typedef struct X509_name_entry_st { ASN1_OBJECT *object; ASN1_BIT_STRING *value; int set; int size; /* temp variable */ } X509_NAME_ENTRY; The size is a temporary variable used by i2d_NAME and set is the set number for the particular NAME_ENTRY. A X509_NAME is encoded as a sequence of sequence of sets. Normally each set contains only a single item. Sometimes it contains more. Normally throughout this library there will be only one item per set. The set field contains the 'set' that this entry is a member of. So if you have just created a X509_NAME structure and populated it with X509_NAME_ENTRYs, you should then traverse the X509_NAME (which is just a STACK) and set the 'set/' field to incrementing numbers. For more details on why this is done, read the ASN.1 spec for Distinguished Names. X509_REQ CertificateRequest X509_REQ_INFO CertificateRequestInfo These are used to hold certificate requests. X509_CRL CertificateRevocationList These are used to hold a certificate revocation list RSAPrivateKey PrivateKeyInfo RSAPublicKey PublicKeyInfo Both these 'function groups' operate on 'RSA' structures (see rsa.doc). The difference is that the RSAPublicKey operations only manipulate the m and e fields in the RSA structure. DSAPrivateKey DSS private key DSAPublicKey DSS public key Both these 'function groups' operate on 'DSS' structures (see dsa.doc). The difference is that the RSAPublicKey operations only manipulate the XXX fields in the DSA structure. DHparams DHParameter This is used to hold the p and g value for The Diffie-Hellman operation. The function deal with the 'DH' strucure (see dh.doc). Now all of these function types can be used with several other functions to give quite useful set of general manipulation routines. Normally one would not uses these functions directly but use them via macros. char *ASN1_dup(int (*i2d)(),char *(*d2i)(),char *x); 'x' is the input structure case to a 'char *', 'i2d' is the 'i2d_TYPE' function for the type that 'x' is and d2i is the 'd2i_TYPE' function for the type that 'x' is. As is obvious from the parameters, this function duplicates the strucutre by transforming it into the DER form and then re-loading it into a new strucutre and returning the new strucutre. This is obviously a bit cpu intensive but when faced with a complex dynamic structure this is the simplest programming approach. There are macros for duplicating the major data types but is simple to add extras. char *ASN1_d2i_fp(char *(*new)(),char *(*d2i)(),FILE *fp,unsigned char **x); 'x' is a pointer to a pointer of the 'desired type'. new and d2i are the corresponding 'TYPE_new' and 'd2i_TYPE' functions for the type and 'fp' is an open file pointer to read from. This function reads from 'fp' as much data as it can and then uses 'd2i' to parse the bytes to load and return the parsed strucutre in 'x' (if it was non-NULL) and to actually return the strucutre. The behavior of 'x' is as per all the other d2i functions. char *ASN1_d2i_bio(char *(*new)(),char *(*d2i)(),BIO *fp,unsigned char **x); The 'BIO' is the new IO type being used in SSLeay (see bio.doc). This function is the same as ASN1_d2i_fp() except for the BIO argument. ASN1_d2i_fp() actually calls this function. int ASN1_i2d_fp(int (*i2d)(),FILE *out,unsigned char *x); 'x' is converted to bytes by 'i2d' and then written to 'out'. ASN1_i2d_fp and ASN1_d2i_fp are not really symetric since ASN1_i2d_fp will read all available data from the file pointer before parsing a single item while ASN1_i2d_fp can be used to write a sequence of data objects. To read a series of objects from a file I would sugest loading the file into a buffer and calling the relevent 'd2i' functions. char *ASN1_d2i_bio(char *(*new)(),char *(*d2i)(),BIO *fp,unsigned char **x); This function is the same as ASN1_i2d_fp() except for the BIO argument. ASN1_i2d_fp() actually calls this function. char * PEM_ASN1_read(char *(*d2i)(),char *name,FILE *fp,char **x,int (*cb)()); This function will read the next PEM encoded (base64) object of the same type as 'x' (loaded by the d2i function). 'name' is the name that is in the '-----BEGIN name-----' that designates the start of that object type. If the data is encrypted, 'cb' will be called to prompt for a password. If it is NULL a default function will be used to prompt from the password. 'x' is delt with as per the standard 'd2i' function interface. This function can be used to read a series of objects from a file. While any data type can be encrypted (see PEM_ASN1_write) only RSA private keys tend to be encrypted. char * PEM_ASN1_read_bio(char *(*d2i)(),char *name,BIO *fp, char **x,int (*cb)()); Same as PEM_ASN1_read() except using a BIO. This is called by PEM_ASN1_read(). int PEM_ASN1_write(int (*i2d)(),char *name,FILE *fp,char *x,EVP_CIPHER *enc, unsigned char *kstr,int klen,int (*callback)()); int PEM_ASN1_write_bio(int (*i2d)(),char *name,BIO *fp, char *x,EVP_CIPHER *enc,unsigned char *kstr,int klen, int (*callback)()); int ASN1_sign(int (*i2d)(), X509_ALGOR *algor1, X509_ALGOR *algor2, ASN1_BIT_STRING *signature, char *data, RSA *rsa, EVP_MD *type); int ASN1_verify(int (*i2d)(), X509_ALGOR *algor1, ASN1_BIT_STRING *signature,char *data, RSA *rsa); int ASN1_BIT_STRING_cmp(ASN1_BIT_STRING *a, ASN1_BIT_STRING *b); ASN1_BIT_STRING *ASN1_BIT_STRING_type_new(int type ); int ASN1_UTCTIME_check(ASN1_UTCTIME *a); void ASN1_UTCTIME_print(BIO *fp,ASN1_UTCTIME *a); ASN1_UTCTIME *ASN1_UTCTIME_dup(ASN1_UTCTIME *a); ASN1_BIT_STRING *d2i_asn1_print_type(ASN1_BIT_STRING **a,unsigned char **pp, long length,int type); int i2d_ASN1_SET(STACK *a, unsigned char **pp, int (*func)(), int ex_tag, int ex_class); STACK * d2i_ASN1_SET(STACK **a, unsigned char **pp, long length, char *(*func)(), int ex_tag, int ex_class); int i2a_ASN1_OBJECT(BIO *bp,ASN1_OBJECT *object); int i2a_ASN1_INTEGER(BIO *bp, ASN1_INTEGER *a); int a2i_ASN1_INTEGER(BIO *bp,ASN1_INTEGER *bs,char *buf,int size); int ASN1_INTEGER_set(ASN1_INTEGER *a, long v); long ASN1_INTEGER_get(ASN1_INTEGER *a); ASN1_INTEGER *BN_to_ASN1_INTEGER(BIGNUM *bn, ASN1_INTEGER *ai); BIGNUM *ASN1_INTEGER_to_BN(ASN1_INTEGER *ai,BIGNUM *bn); /* given a string, return the correct type. Max is the maximum number * of bytes to parse. It stops parsing when 'max' bytes have been * processed or a '\0' is hit */ int ASN1_PRINTABLE_type(unsigned char *s,int max); void ASN1_parse(BIO *fp,unsigned char *pp,long len); int i2d_ASN1_bytes(ASN1_BIT_STRING *a, unsigned char **pp, int tag, int class); ASN1_BIT_STRING *d2i_ASN1_bytes(ASN1_OCTET_STRING **a, unsigned char **pp, long length, int Ptag, int Pclass); /* PARSING */ int asn1_Finish(ASN1_CTX *c); /* SPECIALS */ int ASN1_get_object(unsigned char **pp, long *plength, int *ptag, int *pclass, long omax); int ASN1_check_infinite_end(unsigned char **p,long len); void ASN1_put_object(unsigned char **pp, int constructed, int length, int tag, int class); int ASN1_object_size(int constructed, int length, int tag); X509 * X509_get_cert(CERTIFICATE_CTX *ctx,X509_NAME * name,X509 *tmp_x509); int X509_add_cert(CERTIFICATE_CTX *ctx,X509 *); char * X509_cert_verify_error_string(int n); int X509_add_cert_file(CERTIFICATE_CTX *c,char *file, int type); char * X509_gmtime (char *s, long adj); int X509_add_cert_dir (CERTIFICATE_CTX *c,char *dir, int type); int X509_load_verify_locations (CERTIFICATE_CTX *ctx, char *file_env, char *dir_env); int X509_set_default_verify_paths(CERTIFICATE_CTX *cts); X509 * X509_new_D2i_X509(int len, unsigned char *p); char * X509_get_default_cert_area(void ); char * X509_get_default_cert_dir(void ); char * X509_get_default_cert_file(void ); char * X509_get_default_cert_dir_env(void ); char * X509_get_default_cert_file_env(void ); char * X509_get_default_private_dir(void ); X509_REQ *X509_X509_TO_req(X509 *x, RSA *rsa); int X509_cert_verify(CERTIFICATE_CTX *ctx,X509 *xs, int (*cb)()); CERTIFICATE_CTX *CERTIFICATE_CTX_new(); void CERTIFICATE_CTX_free(CERTIFICATE_CTX *c); void X509_NAME_print(BIO *fp, X509_NAME *name, int obase); int X509_print_fp(FILE *fp,X509 *x); int X509_print(BIO *fp,X509 *x); X509_INFO * X509_INFO_new(void); void X509_INFO_free(X509_INFO *a); char * X509_NAME_oneline(X509_NAME *a); #define X509_verify(x,rsa) #define X509_REQ_verify(x,rsa) #define X509_CRL_verify(x,rsa) #define X509_sign(x,rsa,md) #define X509_REQ_sign(x,rsa,md) #define X509_CRL_sign(x,rsa,md) #define X509_dup(x509) #define d2i_X509_fp(fp,x509) #define i2d_X509_fp(fp,x509) #define d2i_X509_bio(bp,x509) #define i2d_X509_bio(bp,x509) #define X509_CRL_dup(crl) #define d2i_X509_CRL_fp(fp,crl) #define i2d_X509_CRL_fp(fp,crl) #define d2i_X509_CRL_bio(bp,crl) #define i2d_X509_CRL_bio(bp,crl) #define X509_REQ_dup(req) #define d2i_X509_REQ_fp(fp,req) #define i2d_X509_REQ_fp(fp,req) #define d2i_X509_REQ_bio(bp,req) #define i2d_X509_REQ_bio(bp,req) #define RSAPrivateKey_dup(rsa) #define d2i_RSAPrivateKey_fp(fp,rsa) #define i2d_RSAPrivateKey_fp(fp,rsa) #define d2i_RSAPrivateKey_bio(bp,rsa) #define i2d_RSAPrivateKey_bio(bp,rsa) #define X509_NAME_dup(xn) #define X509_NAME_ENTRY_dup(ne) void X509_REQ_print_fp(FILE *fp,X509_REQ *req); void X509_REQ_print(BIO *fp,X509_REQ *req); RSA *X509_REQ_extract_key(X509_REQ *req); RSA *X509_extract_key(X509 *x509); int X509_issuer_and_serial_cmp(X509 *a, X509 *b); unsigned long X509_issuer_and_serial_hash(X509 *a); X509_NAME * X509_get_issuer_name(X509 *a); int X509_issuer_name_cmp(X509 *a, X509 *b); unsigned long X509_issuer_name_hash(X509 *a); X509_NAME * X509_get_subject_name(X509 *a); int X509_subject_name_cmp(X509 *a,X509 *b); unsigned long X509_subject_name_hash(X509 *x); int X509_NAME_cmp (X509_NAME *a, X509_NAME *b); unsigned long X509_NAME_hash(X509_NAME *x); ==== bio.doc ======================================================== BIO Routines This documentation is rather sparse, you are probably best off looking at the code for specific details. The BIO library is a IO abstraction that was originally inspired by the need to have callbacks to perform IO to FILE pointers when using Windows 3.1 DLLs. There are two types of BIO; a source/sink type and a filter type. The source/sink methods are as follows: - BIO_s_mem() memory buffer - a read/write byte array that grows until memory runs out :-). - BIO_s_file() FILE pointer - A wrapper around the normal 'FILE *' commands, good for use with stdin/stdout. - BIO_s_fd() File descriptor - A wrapper around file descriptors, often used with pipes. - BIO_s_socket() Socket - Used around sockets. It is mostly in the Microsoft world that sockets are different from file descriptors and there are all those ugly winsock commands. - BIO_s_null() Null - read nothing and write nothing.; a useful endpoint for filter type BIO's specifically things like the message digest BIO. The filter types are - BIO_f_buffer() IO buffering - does output buffering into larger chunks and performs input buffering to allow gets() type functions. - BIO_f_md() Message digest - a transparent filter that can be asked to return a message digest for the data that has passed through it. - BIO_f_cipher() Encrypt or decrypt all data passing through the filter. - BIO_f_base64() Base64 decode on read and encode on write. - BIO_f_ssl() A filter that performs SSL encryption on the data sent through it. Base BIO functions. The BIO library has a set of base functions that are implemented for each particular type. Filter BIOs will normally call the equivalent function on the source/sink BIO that they are layered on top of after they have performed some modification to the data stream. Multiple filter BIOs can be 'push' into a stack of modifers, so to read from a file, unbase64 it, then decrypt it, a BIO_f_cipher, BIO_f_base64 and a BIO_s_file would probably be used. If a sha-1 and md5 message digest needed to be generated, a stack two BIO_f_md() BIOs and a BIO_s_null() BIO could be used. The base functions are - BIO *BIO_new(BIO_METHOD *type); Create a new BIO of type 'type'. - int BIO_free(BIO *a); Free a BIO structure. Depending on the configuration, this will free the underlying data object for a source/sink BIO. - int BIO_read(BIO *b, char *data, int len); Read upto 'len' bytes into 'data'. - int BIO_gets(BIO *bp,char *buf, int size); Depending on the BIO, this can either be a 'get special' or a get one line of data, as per fgets(); - int BIO_write(BIO *b, char *data, int len); Write 'len' bytes from 'data' to the 'b' BIO. - int BIO_puts(BIO *bp,char *buf); Either a 'put special' or a write null terminated string as per fputs(). - long BIO_ctrl(BIO *bp,int cmd,long larg,char *parg); A control function which is used to manipulate the BIO structure and modify it's state and or report on it. This function is just about never used directly, rather it should be used in conjunction with BIO_METHOD specific macros. - BIO *BIO_push(BIO *new_top, BIO *old); new_top is apped to the top of the 'old' BIO list. new_top should be a filter BIO. All writes will go through 'new_top' first and last on read. 'old' is returned. - BIO *BIO_pop(BIO *bio); the new topmost BIO is returned, NULL if there are no more. If a particular low level BIO method is not supported (normally BIO_gets()), -2 will be returned if that method is called. Otherwise the IO methods (read, write, gets, puts) will return the number of bytes read or written, and 0 or -1 for error (or end of input). For the -1 case, BIO_should_retry(bio) can be called to determine if it was a genuine error or a temporary problem. -2 will also be returned if the BIO has not been initalised yet, in all cases, the correct error codes are set (accessible via the ERR library). The following functions are convenience functions: - int BIO_printf(BIO *bio, char * format, ..); printf but to a BIO handle. - long BIO_ctrl_int(BIO *bp,int cmd,long larg,int iarg); a convenience function to allow a different argument types to be passed to BIO_ctrl(). - int BIO_dump(BIO *b,char *bytes,int len); output 'len' bytes from 'bytes' in a hex dump debug format. - long BIO_debug_callback(BIO *bio, int cmd, char *argp, int argi, long argl, long ret) - a default debug BIO callback, this is mentioned below. To use this one normally has to use the BIO_set_callback_arg() function to assign an output BIO for the callback to use. - BIO *BIO_find_type(BIO *bio,int type); when there is a 'stack' of BIOs, this function scan the list and returns the first that is of type 'type', as listed in buffer.h under BIO_TYPE_XXX. - void BIO_free_all(BIO *bio); Free the bio and all other BIOs in the list. It walks the bio->next_bio list. Extra commands are normally implemented as macros calling BIO_ctrl(). - BIO_number_read(BIO *bio) - the number of bytes processed by BIO_read(bio,.). - BIO_number_written(BIO *bio) - the number of bytes written by BIO_write(bio,.). - BIO_reset(BIO *bio) - 'reset' the BIO. - BIO_eof(BIO *bio) - non zero if we are at the current end of input. - BIO_set_close(BIO *bio, int close_flag) - set the close flag. - BIO_get_close(BIO *bio) - return the close flag. BIO_pending(BIO *bio) - return the number of bytes waiting to be read (normally buffered internally). - BIO_flush(BIO *bio) - output any data waiting to be output. - BIO_should_retry(BIO *io) - after a BIO_read/BIO_write operation returns 0 or -1, a call to this function will return non zero if you should retry the call later (this is for non-blocking IO). - BIO_should_read(BIO *io) - we should retry when data can be read. - BIO_should_write(BIO *io) - we should retry when data can be written. - BIO_method_name(BIO *io) - return a string for the method name. - BIO_method_type(BIO *io) - return the unique ID of the BIO method. - BIO_set_callback(BIO *io, long (*callback)(BIO *io, int cmd, char *argp, int argi, long argl, long ret); - sets the debug callback. - BIO_get_callback(BIO *io) - return the assigned function as mentioned above. - BIO_set_callback_arg(BIO *io, char *arg) - assign some data against the BIO. This is normally used by the debug callback but could in reality be used for anything. To get an idea of how all this works, have a look at the code in the default debug callback mentioned above. The callback can modify the return values. Details of the BIO_METHOD structure. typedef struct bio_method_st { int type; char *name; int (*bwrite)(); int (*bread)(); int (*bputs)(); int (*bgets)(); long (*ctrl)(); int (*create)(); int (*destroy)(); } BIO_METHOD; The 'type' is the numeric type of the BIO, these are listed in buffer.h; 'Name' is a textual representation of the BIO 'type'. The 7 function pointers point to the respective function methods, some of which can be NULL if not implemented. The BIO structure typedef struct bio_st { BIO_METHOD *method; long (*callback)(BIO * bio, int mode, char *argp, int argi, long argl, long ret); char *cb_arg; /* first argument for the callback */ int init; int shutdown; int flags; /* extra storage */ int num; char *ptr; struct bio_st *next_bio; /* used by filter BIOs */ int references; unsigned long num_read; unsigned long num_write; } BIO; - 'Method' is the BIO method. - 'callback', when configured, is called before and after each BIO method is called for that particular BIO. This is intended primarily for debugging and of informational feedback. - 'init' is 0 when the BIO can be used for operation. Often, after a BIO is created, a number of operations may need to be performed before it is available for use. An example is for BIO_s_sock(). A socket needs to be assigned to the BIO before it can be used. - 'shutdown', this flag indicates if the underlying communication primitive being used should be closed/freed when the BIO is closed. - 'flags' is used to hold extra state. It is primarily used to hold information about why a non-blocking operation failed and to record startup protocol information for the SSL BIO. - 'num' and 'ptr' are used to hold instance specific state like file descriptors or local data structures. - 'next_bio' is used by filter BIOs to hold the pointer of the next BIO in the chain. written data is sent to this BIO and data read is taken from it. - 'references' is used to indicate the number of pointers to this structure. This needs to be '1' before a call to BIO_free() is made if the BIO_free() function is to actually free() the structure, otherwise the reference count is just decreased. The actual BIO subsystem does not really use this functionality but it is useful when used in more advanced applicaion. - num_read and num_write are the total number of bytes read/written via the 'read()' and 'write()' methods. BIO_ctrl operations. The following is the list of standard commands passed as the second parameter to BIO_ctrl() and should be supported by all BIO as best as possible. Some are optional, some are manditory, in any case, where is makes sense, a filter BIO should pass such requests to underlying BIO's. - BIO_CTRL_RESET - Reset the BIO back to an initial state. - BIO_CTRL_EOF - return 0 if we are not at the end of input, non 0 if we are. - BIO_CTRL_INFO - BIO specific special command, normal information return. - BIO_CTRL_SET - set IO specific parameter. - BIO_CTRL_GET - get IO specific parameter. - BIO_CTRL_GET_CLOSE - Get the close on BIO_free() flag, one of BIO_CLOSE or BIO_NOCLOSE. - BIO_CTRL_SET_CLOSE - Set the close on BIO_free() flag. - BIO_CTRL_PENDING - Return the number of bytes available for instant reading - BIO_CTRL_FLUSH - Output pending data, return number of bytes output. - BIO_CTRL_SHOULD_RETRY - After an IO error (-1 returned) should we 'retry' when IO is possible on the underlying IO object. - BIO_CTRL_RETRY_TYPE - What kind of IO are we waiting on. The following command is a special BIO_s_file() specific option. - BIO_CTRL_SET_FILENAME - specify a file to open for IO. The BIO_CTRL_RETRY_TYPE needs a little more explanation. When performing non-blocking IO, or say reading on a memory BIO, when no data is present (or cannot be written), BIO_read() and/or BIO_write() will return -1. BIO_should_retry(bio) will return true if this is due to an IO condition rather than an actual error. In the case of BIO_s_mem(), a read when there is no data will return -1 and a should retry when there is more 'read' data. The retry type is deduced from 2 macros BIO_should_read(bio) and BIO_should_write(bio). Now while it may appear obvious that a BIO_read() failure should indicate that a retry should be performed when more read data is available, this is often not true when using things like an SSL BIO. During the SSL protocol startup multiple reads and writes are performed, triggered by any SSL_read or SSL_write. So to write code that will transparently handle either a socket or SSL BIO, i=BIO_read(bio,..) if (I == -1) { if (BIO_should_retry(bio)) { if (BIO_should_read(bio)) { /* call us again when BIO can be read */ } if (BIO_should_write(bio)) { /* call us again when BIO can be written */ } } } At this point in time only read and write conditions can be used but in the future I can see the situation for other conditions, specifically with SSL there could be a condition of a X509 certificate lookup taking place and so the non- blocking BIO_read would require a retry when the certificate lookup subsystem has finished it's lookup. This is all makes more sense and is easy to use in a event loop type setup. When using the SSL BIO, either SSL_read() or SSL_write()s can be called during the protocol startup and things will still work correctly. The nice aspect of the use of the BIO_should_retry() macro is that all the errno codes that indicate a non-fatal error are encapsulated in one place. The Windows specific error codes and WSAGetLastError() calls are also hidden from the application. Notes on each BIO method. Normally buffer.h is just required but depending on the BIO_METHOD, ssl.h or evp.h will also be required. BIO_METHOD *BIO_s_mem(void); - BIO_set_mem_buf(BIO *bio, BUF_MEM *bm, int close_flag) - set the underlying BUF_MEM structure for the BIO to use. - BIO_get_mem_ptr(BIO *bio, char **pp) - if pp is not NULL, set it to point to the memory array and return the number of bytes available. A read/write BIO. Any data written is appended to the memory array and any read is read from the front. This BIO can be used for read/write at the same time. BIO_gets() is supported in the fgets() sense. BIO_CTRL_INFO can be used to retrieve pointers to the memory buffer and it's length. BIO_METHOD *BIO_s_file(void); - BIO_set_fp(BIO *bio, FILE *fp, int close_flag) - set 'FILE *' to use. - BIO_get_fp(BIO *bio, FILE **fp) - get the 'FILE *' in use. - BIO_read_filename(BIO *bio, char *name) - read from file. - BIO_write_filename(BIO *bio, char *name) - write to file. - BIO_append_filename(BIO *bio, char *name) - append to file. This BIO sits over the normal system fread()/fgets() type functions. Gets() is supported. This BIO in theory could be used for read and write but it is best to think of each BIO of this type as either a read or a write BIO, not both. BIO_METHOD *BIO_s_socket(void); BIO_METHOD *BIO_s_fd(void); - BIO_sock_should_retry(int i) - the underlying function used to determine if a call should be retried; the argument is the '0' or '-1' returned by the previous BIO operation. - BIO_fd_should_retry(int i) - same as the - BIO_sock_should_retry() except that it is different internally. - BIO_set_fd(BIO *bio, int fd, int close_flag) - set the file descriptor to use - BIO_get_fd(BIO *bio, int *fd) - get the file descriptor. These two methods are very similar. Gets() is not supported, if you want this functionality, put a BIO_f_buffer() onto it. This BIO is bi-directional if the underlying file descriptor is. This is normally the case for sockets but not the case for stdio descriptors. BIO_METHOD *BIO_s_null(void); Read and write as much data as you like, it all disappears into this BIO. BIO_METHOD *BIO_f_buffer(void); - BIO_get_buffer_num_lines(BIO *bio) - return the number of complete lines in the buffer. - BIO_set_buffer_size(BIO *bio, long size) - set the size of the buffers. This type performs input and output buffering. It performs both at the same time. The size of the buffer can be set via the set buffer size option. Data buffered for output is only written when the buffer fills. BIO_METHOD *BIO_f_ssl(void); - BIO_set_ssl(BIO *bio, SSL *ssl, int close_flag) - the SSL structure to use. - BIO_get_ssl(BIO *bio, SSL **ssl) - get the SSL structure in use. The SSL bio is a little different from normal BIOs because the underlying SSL structure is a little different. A SSL structure performs IO via a read and write BIO. These can be different and are normally set via the SSL_set_rbio()/SSL_set_wbio() calls. The SSL_set_fd() calls are just wrappers that create socket BIOs and then call SSL_set_bio() where the read and write BIOs are the same. The BIO_push() operation makes the SSLs IO BIOs the same, so make sure the BIO pushed is capable of two directional traffic. If it is not, you will have to install the BIOs via the more conventional SSL_set_bio() call. BIO_pop() will retrieve the 'SSL read' BIO. BIO_METHOD *BIO_f_md(void); - BIO_set_md(BIO *bio, EVP_MD *md) - set the message digest to use. - BIO_get_md(BIO *bio, EVP_MD **mdp) - return the digest method in use in mdp, return 0 if not set yet. - BIO_reset() reinitializes the digest (EVP_DigestInit()) and passes the reset to the underlying BIOs. All data read or written via BIO_read() or BIO_write() to this BIO will be added to the calculated digest. This implies that this BIO is only one directional. If read and write operations are performed, two separate BIO_f_md() BIOs are reuqired to generate digests on both the input and the output. BIO_gets(BIO *bio, char *md, int size) will place the generated digest into 'md' and return the number of bytes. The EVP_MAX_MD_SIZE should probably be used to size the 'md' array. Reading the digest will also reset it. BIO_METHOD *BIO_f_cipher(void); - BIO_reset() reinitializes the cipher. - BIO_flush() should be called when the last bytes have been output to flush the final block of block ciphers. - BIO_get_cipher_status(BIO *b), when called after the last read from a cipher BIO, returns non-zero if the data decrypted correctly, otherwise, 0. - BIO_set_cipher(BIO *b, EVP_CIPHER *c, unsigned char *key, unsigned char *iv, int encrypt) This function is used to setup a cipher BIO. The length of key and iv are specified by the choice of EVP_CIPHER. Encrypt is 1 to encrypt and 0 to decrypt. BIO_METHOD *BIO_f_base64(void); - BIO_flush() should be called when the last bytes have been output. This BIO base64 encodes when writing and base64 decodes when reading. It will scan the input until a suitable begin line is found. After reading data, BIO_reset() will reset the BIO to start scanning again. Do not mix reading and writing on the same base64 BIO. It is meant as a single stream BIO. Directions type both BIO_s_mem() one/both BIO_s_file() both BIO_s_fd() both BIO_s_socket() both BIO_s_null() both BIO_f_buffer() one BIO_f_md() one BIO_f_cipher() one BIO_f_base64() both BIO_f_ssl() It is easy to mix one and two directional BIOs, all one has to do is to keep two separate BIO pointers for reading and writing and be careful about usage of underlying BIOs. The SSL bio by it's very nature has to be two directional but the BIO_push() command will push the one BIO into the SSL BIO for both reading and writing. The best example program to look at is apps/enc.c and/or perhaps apps/dgst.c. ==== blowfish.doc ======================================================== The Blowfish library. Blowfish is a block cipher that operates on 64bit (8 byte) quantities. It uses variable size key, but 128bit (16 byte) key would normally be considered good. It can be used in all the modes that DES can be used. This library implements the ecb, cbc, cfb64, ofb64 modes. Blowfish is quite a bit faster that DES, and much faster than IDEA or RC2. It is one of the faster block ciphers. For all calls that have an 'input' and 'output' variables, they can be the same. This library requires the inclusion of 'blowfish.h'. All of the encryption functions take what is called an BF_KEY as an argument. An BF_KEY is an expanded form of the Blowfish key. For all modes of the Blowfish algorithm, the BF_KEY used for decryption is the same one that was used for encryption. The define BF_ENCRYPT is passed to specify encryption for the functions that require an encryption/decryption flag. BF_DECRYPT is passed to specify decryption. Please note that any of the encryption modes specified in my DES library could be used with Blowfish. I have only implemented ecb, cbc, cfb64 and ofb64 for the following reasons. - ecb is the basic Blowfish encryption. - cbc is the normal 'chaining' form for block ciphers. - cfb64 can be used to encrypt single characters, therefore input and output do not need to be a multiple of 8. - ofb64 is similar to cfb64 but is more like a stream cipher, not as secure (not cipher feedback) but it does not have an encrypt/decrypt mode. - If you want triple Blowfish, thats 384 bits of key and you must be totally obsessed with security. Still, if you want it, it is simple enough to copy the function from the DES library and change the des_encrypt to BF_encrypt; an exercise left for the paranoid reader :-). The functions are as follows: void BF_set_key( BF_KEY *ks; int len; unsigned char *key; BF_set_key converts an 'len' byte key into a BF_KEY. A 'ks' is an expanded form of the 'key' which is used to perform actual encryption. It can be regenerated from the Blowfish key so it only needs to be kept when encryption or decryption is about to occur. Don't save or pass around BF_KEY's since they are CPU architecture dependent, 'key's are not. Blowfish is an interesting cipher in that it can be used with a variable length key. 'len' is the length of 'key' to be used as the key. A 'len' of 16 is recomended by me, but blowfish can use upto 72 bytes. As a warning, blowfish has a very very slow set_key function, it actually runs BF_encrypt 521 times. void BF_encrypt(unsigned long *data, BF_KEY *key); void BF_decrypt(unsigned long *data, BF_KEY *key); These are the Blowfish encryption function that gets called by just about every other Blowfish routine in the library. You should not use this function except to implement 'modes' of Blowfish. I say this because the functions that call this routine do the conversion from 'char *' to long, and this needs to be done to make sure 'non-aligned' memory access do not occur. Data is a pointer to 2 unsigned long's and key is the BF_KEY to use. void BF_ecb_encrypt( unsigned char *in, unsigned char *out, BF_KEY *key, int encrypt); This is the basic Electronic Code Book form of Blowfish (in DES this mode is called Electronic Code Book so I'm going to use the term for blowfish as well. Input is encrypted into output using the key represented by key. Depending on the encrypt, encryption or decryption occurs. Input is 8 bytes long and output is 8 bytes. void BF_cbc_encrypt( unsigned char *in, unsigned char *out, long length, BF_KEY *ks, unsigned char *ivec, int encrypt); This routine implements Blowfish in Cipher Block Chaining mode. Input, which should be a multiple of 8 bytes is encrypted (or decrypted) to output which will also be a multiple of 8 bytes. The number of bytes is in length (and from what I've said above, should be a multiple of 8). If length is not a multiple of 8, bad things will probably happen. ivec is the initialisation vector. This function updates iv after each call so that it can be passed to the next call to BF_cbc_encrypt(). void BF_cfb64_encrypt( unsigned char *in, unsigned char *out, long length, BF_KEY *schedule, unsigned char *ivec, int *num, int encrypt); This is one of the more useful functions in this Blowfish library, it implements CFB mode of Blowfish with 64bit feedback. This allows you to encrypt an arbitrary number of bytes, you do not require 8 byte padding. Each call to this routine will encrypt the input bytes to output and then update ivec and num. Num contains 'how far' we are though ivec. 'Encrypt' is used to indicate encryption or decryption. CFB64 mode operates by using the cipher to generate a stream of bytes which is used to encrypt the plain text. The cipher text is then encrypted to generate the next 64 bits to be xored (incrementally) with the next 64 bits of plain text. As can be seen from this, to encrypt or decrypt, the same 'cipher stream' needs to be generated but the way the next block of data is gathered for encryption is different for encryption and decryption. void BF_ofb64_encrypt( unsigned char *in, unsigned char *out, long length, BF_KEY *schedule, unsigned char *ivec, int *num); This functions implements OFB mode of Blowfish with 64bit feedback. This allows you to encrypt an arbitrary number of bytes, you do not require 8 byte padding. Each call to this routine will encrypt the input bytes to output and then update ivec and num. Num contains 'how far' we are though ivec. This is in effect a stream cipher, there is no encryption or decryption mode. For reading passwords, I suggest using des_read_pw_string() from my DES library. To generate a password from a text string, I suggest using MD5 (or MD2) to produce a 16 byte message digest that can then be passed directly to BF_set_key(). ===== For more information about the specific Blowfish modes in this library (ecb, cbc, cfb and ofb), read the section entitled 'Modes of DES' from the documentation on my DES library. What is said about DES is directly applicable for Blowfish. ==== bn.doc ======================================================== The Big Number library. #include "bn.h" when using this library. This big number library was written for use in implementing the RSA and DH public key encryption algorithms. As such, features such as negative numbers have not been extensively tested but they should work as expected. This library uses dynamic memory allocation for storing its data structures and so there are no limit on the size of the numbers manipulated by these routines but there is always the requirement to check return codes from functions just in case a memory allocation error has occurred. The basic object in this library is a BIGNUM. It is used to hold a single large integer. This type should be considered opaque and fields should not be modified or accessed directly. typedef struct bignum_st { int top; /* Index of last used d. */ BN_ULONG *d; /* Pointer to an array of 'BITS2' bit chunks. */ int max; /* Size of the d array. */ int neg; } BIGNUM; The big number is stored in a malloced array of BN_ULONG's. A BN_ULONG can be either 16, 32 or 64 bits in size, depending on the 'number of bits' specified in bn.h. The 'd' field is this array. 'max' is the size of the 'd' array that has been allocated. 'top' is the 'last' entry being used, so for a value of 4, bn.d[0]=4 and bn.top=1. 'neg' is 1 if the number is negative. When a BIGNUM is '0', the 'd' field can be NULL and top == 0. Various routines in this library require the use of 'temporary' BIGNUM variables during their execution. Due to the use of dynamic memory allocation to create BIGNUMs being rather expensive when used in conjunction with repeated subroutine calls, the BN_CTX structure is used. This structure contains BN_CTX BIGNUMs. BN_CTX is the maximum number of temporary BIGNUMs any publicly exported function will use. #define BN_CTX 12 typedef struct bignum_ctx { int tos; /* top of stack */ BIGNUM *bn[BN_CTX]; /* The variables */ } BN_CTX; The functions that follow have been grouped according to function. Most arithmetic functions return a result in the first argument, sometimes this first argument can also be an input parameter, sometimes it cannot. These restrictions are documented. extern BIGNUM *BN_value_one; There is one variable defined by this library, a BIGNUM which contains the number 1. This variable is useful for use in comparisons and assignment. Get Size functions. int BN_num_bits(BIGNUM *a); This function returns the size of 'a' in bits. int BN_num_bytes(BIGNUM *a); This function (macro) returns the size of 'a' in bytes. For conversion of BIGNUMs to byte streams, this is the number of bytes the output string will occupy. If the output byte format specifies that the 'top' bit indicates if the number is signed, so an extra '0' byte is required if the top bit on a positive number is being written, it is upto the application to make this adjustment. Like I said at the start, I don't really support negative numbers :-). Creation/Destruction routines. BIGNUM *BN_new(); Return a new BIGNUM object. The number initially has a value of 0. If there is an error, NULL is returned. void BN_free(BIGNUM *a); Free()s a BIGNUM. void BN_clear(BIGNUM *a); Sets 'a' to a value of 0 and also zeros all unused allocated memory. This function is used to clear a variable of 'sensitive' data that was held in it. void BN_clear_free(BIGNUM *a); This function zeros the memory used by 'a' and then free()'s it. This function should be used to BN_free() BIGNUMS that have held sensitive numeric values like RSA private key values. Both this function and BN_clear tend to only be used by RSA and DH routines. BN_CTX *BN_CTX_new(void); Returns a new BN_CTX. NULL on error. void BN_CTX_free(BN_CTX *c); Free a BN_CTX structure. The BIGNUMs in 'c' are BN_clear_free()ed. BIGNUM *bn_expand(BIGNUM *b, int bits); This is an internal function that should not normally be used. It ensures that 'b' has enough room for a 'bits' bit number. It is mostly used by the various BIGNUM routines. If there is an error, NULL is returned. if not, 'b' is returned. BIGNUM *BN_copy(BIGNUM *to, BIGNUM *from); The 'from' is copied into 'to'. NULL is returned if there is an error, otherwise 'to' is returned. BIGNUM *BN_dup(BIGNUM *a); A new BIGNUM is created and returned containing the value of 'a'. NULL is returned on error. Comparison and Test Functions. int BN_is_zero(BIGNUM *a) Return 1 if 'a' is zero, else 0. int BN_is_one(a) Return 1 is 'a' is one, else 0. int BN_is_word(a,w) Return 1 if 'a' == w, else 0. 'w' is a BN_ULONG. int BN_cmp(BIGNUM *a, BIGNUM *b); Return -1 if 'a' is less than 'b', 0 if 'a' and 'b' are the same and 1 is 'a' is greater than 'b'. This is a signed comparison. int BN_ucmp(BIGNUM *a, BIGNUM *b); This function is the same as BN_cmp except that the comparison ignores the sign of the numbers. Arithmetic Functions For all of these functions, 0 is returned if there is an error and 1 is returned for success. The return value should always be checked. eg. if (!BN_add(r,a,b)) goto err; Unless explicitly mentioned, the 'return' value can be one of the 'parameters' to the function. int BN_add(BIGNUM *r, BIGNUM *a, BIGNUM *b); Add 'a' and 'b' and return the result in 'r'. This is r=a+b. int BN_sub(BIGNUM *r, BIGNUM *a, BIGNUM *b); Subtract 'a' from 'b' and put the result in 'r'. This is r=a-b. int BN_lshift(BIGNUM *r, BIGNUM *a, int n); Shift 'a' left by 'n' bits. This is r=a*(2^n). int BN_lshift1(BIGNUM *r, BIGNUM *a); Shift 'a' left by 1 bit. This form is more efficient than BN_lshift(r,a,1). This is r=a*2. int BN_rshift(BIGNUM *r, BIGNUM *a, int n); Shift 'a' right by 'n' bits. This is r=int(a/(2^n)). int BN_rshift1(BIGNUM *r, BIGNUM *a); Shift 'a' right by 1 bit. This form is more efficient than BN_rshift(r,a,1). This is r=int(a/2). int BN_mul(BIGNUM *r, BIGNUM *a, BIGNUM *b); Multiply a by b and return the result in 'r'. 'r' must not be either 'a' or 'b'. It has to be a different BIGNUM. This is r=a*b. int BN_sqr(BIGNUM *r, BIGNUM *a, BN_CTX *ctx); Multiply a by a and return the result in 'r'. 'r' must not be 'a'. This function is alot faster than BN_mul(r,a,a). This is r=a*a. int BN_div(BIGNUM *dv, BIGNUM *rem, BIGNUM *m, BIGNUM *d, BN_CTX *ctx); Divide 'm' by 'd' and return the result in 'dv' and the remainder in 'rem'. Either of 'dv' or 'rem' can be NULL in which case that value is not returned. 'ctx' needs to be passed as a source of temporary BIGNUM variables. This is dv=int(m/d), rem=m%d. int BN_mod(BIGNUM *rem, BIGNUM *m, BIGNUM *d, BN_CTX *ctx); Find the remainder of 'm' divided by 'd' and return it in 'rem'. 'ctx' holds the temporary BIGNUMs required by this function. This function is more efficient than BN_div(NULL,rem,m,d,ctx); This is rem=m%d. int BN_mod_mul(BIGNUM *r, BIGNUM *a, BIGNUM *b, BIGNUM *m,BN_CTX *ctx); Multiply 'a' by 'b' and return the remainder when divided by 'm'. 'ctx' holds the temporary BIGNUMs required by this function. This is r=(a*b)%m. int BN_mod_exp(BIGNUM *r, BIGNUM *a, BIGNUM *p, BIGNUM *m,BN_CTX *ctx); Raise 'a' to the 'p' power and return the remainder when divided by 'm'. 'ctx' holds the temporary BIGNUMs required by this function. This is r=(a^p)%m. int BN_reciprocal(BIGNUM *r, BIGNUM *m, BN_CTX *ctx); Return the reciprocal of 'm'. 'ctx' holds the temporary variables required. This function returns -1 on error, otherwise it returns the number of bits 'r' is shifted left to make 'r' into an integer. This number of bits shifted is required in BN_mod_mul_reciprocal(). This is r=(1/m)<<(BN_num_bits(m)+1). int BN_mod_mul_reciprocal(BIGNUM *r, BIGNUM *x, BIGNUM *y, BIGNUM *m, BIGNUM *i, int nb, BN_CTX *ctx); This function is used to perform an efficient BN_mod_mul() operation. If one is going to repeatedly perform BN_mod_mul() with the same modulus is worth calculating the reciprocal of the modulus and then using this function. This operation uses the fact that a/b == a*r where r is the reciprocal of b. On modern computers multiplication is very fast and big number division is very slow. 'x' is multiplied by 'y' and then divided by 'm' and the remainder is returned. 'i' is the reciprocal of 'm' and 'nb' is the number of bits as returned from BN_reciprocal(). Normal usage is as follows. bn=BN_reciprocal(i,m); for (...) { BN_mod_mul_reciprocal(r,x,y,m,i,bn,ctx); } This is r=(x*y)%m. Internally it is approximately r=(x*y)-m*(x*y/m) or r=(x*y)-m*((x*y*i) >> bn) This function is used in BN_mod_exp() and BN_is_prime(). Assignment Operations int BN_one(BIGNUM *a) Set 'a' to hold the value one. This is a=1. int BN_zero(BIGNUM *a) Set 'a' to hold the value zero. This is a=0. int BN_set_word(BIGNUM *a, unsigned long w); Set 'a' to hold the value of 'w'. 'w' is an unsigned long. This is a=w. unsigned long BN_get_word(BIGNUM *a); Returns 'a' in an unsigned long. Not remarkably, often 'a' will be bigger than a word, in which case 0xffffffffL is returned. Word Operations These functions are much more efficient that the normal bignum arithmetic operations. BN_ULONG BN_mod_word(BIGNUM *a, unsigned long w); Return the remainder of 'a' divided by 'w'. This is return(a%w). int BN_add_word(BIGNUM *a, unsigned long w); Add 'w' to 'a'. This function does not take the sign of 'a' into account. This is a+=w; Bit operations. int BN_is_bit_set(BIGNUM *a, int n); This function return 1 if bit 'n' is set in 'a' else 0. int BN_set_bit(BIGNUM *a, int n); This function sets bit 'n' to 1 in 'a'. This is a&= ~(1<<n); int BN_clear_bit(BIGNUM *a, int n); This function sets bit 'n' to zero in 'a'. Return 0 if less than 'n' bits in 'a' else 1. This is a&= ~(1<<n); int BN_mask_bits(BIGNUM *a, int n); Truncate 'a' to n bits long. This is a&= ~((~0)<<n) Format conversion routines. BIGNUM *BN_bin2bn(unsigned char *s, int len,BIGNUM *ret); This function converts 'len' bytes in 's' into a BIGNUM which is put in 'ret'. If ret is NULL, a new BIGNUM is created. Either this new BIGNUM or ret is returned. The number is assumed to be in bigendian form in 's'. By this I mean that to 'ret' is created as follows for 'len' == 5. ret = s[0]*2^32 + s[1]*2^24 + s[2]*2^16 + s[3]*2^8 + s[4]; This function cannot be used to convert negative numbers. It is always assumed the number is positive. The application needs to diddle the 'neg' field of th BIGNUM its self. The better solution would be to save the numbers in ASN.1 format since this is a defined standard for storing big numbers. Look at the functions ASN1_INTEGER *BN_to_ASN1_INTEGER(BIGNUM *bn, ASN1_INTEGER *ai); BIGNUM *ASN1_INTEGER_to_BN(ASN1_INTEGER *ai,BIGNUM *bn); int i2d_ASN1_INTEGER(ASN1_INTEGER *a,unsigned char **pp); ASN1_INTEGER *d2i_ASN1_INTEGER(ASN1_INTEGER **a,unsigned char **pp, long length; int BN_bn2bin(BIGNUM *a, unsigned char *to); This function converts 'a' to a byte string which is put into 'to'. The representation is big-endian in that the most significant byte of 'a' is put into to[0]. This function returns the number of bytes used to hold 'a'. BN_num_bytes(a) would return the same value and can be used to determine how large 'to' needs to be. If the number is negative, this information is lost. Since this library was written to manipulate large positive integers, the inability to save and restore them is not considered to be a problem by me :-). As for BN_bin2bn(), look at the ASN.1 integer encoding funtions for SSLeay. They use BN_bin2bn() and BN_bn2bin() internally. char *BN_bn2ascii(BIGNUM *a); This function returns a malloc()ed string that contains the ascii hexadecimal encoding of 'a'. The number is in bigendian format with a '-' in front if the number is negative. int BN_ascii2bn(BIGNUM **bn, char *a); The inverse of BN_bn2ascii. The function returns the number of characters from 'a' were processed in generating a the bignum. error is inticated by 0 being returned. The number is a hex digit string, optionally with a leading '-'. If *bn is null, a BIGNUM is created and returned via that variable. int BN_print_fp(FILE *fp, BIGNUM *a); 'a' is printed to file pointer 'fp'. It is in the same format that is output from BN_bn2ascii(). 0 is returned on error, 1 if things are ok. int BN_print(BIO *bp, BIGNUM *a); Same as BN_print except that the output is done to the SSLeay libraries BIO routines. BN_print_fp() actually calls this function. Miscellaneous Routines. int BN_rand(BIGNUM *rnd, int bits, int top, int bottom); This function returns in 'rnd' a random BIGNUM that is bits long. If bottom is 1, the number returned is odd. If top is set, the top 2 bits of the number are set. This is useful because if this is set, 2 'n; bit numbers multiplied together will return a 2n bit number. If top was not set, they could produce a 2n-1 bit number. BIGNUM *BN_mod_inverse(BIGNUM *a, BIGNUM *n,BN_CTX *ctx); This function create a new BIGNUM and returns it. This number is the inverse mod 'n' of 'a'. By this it is meant that the returned value 'r' satisfies (a*r)%n == 1. This function is used in the generation of RSA keys. 'ctx', as per usual, is used to hold temporary variables that are required by the function. NULL is returned on error. int BN_gcd(BIGNUM *r,BIGNUM *a,BIGNUM *b,BN_CTX *ctx); 'r' has the greatest common divisor of 'a' and 'b'. 'ctx' is used for temporary variables and 0 is returned on error. int BN_is_prime(BIGNUM *p,int nchecks,void (*callback)(),BN_CTX *ctx, char *cb_arg); This function is used to check if a BIGNUM ('p') is prime. It performs this test by using the Miller-Rabin randomised primality test. This is a probalistic test that requires a number of rounds to ensure the number is prime to a high degree of probability. Since this can take quite some time, a callback function can be passed and it will be called each time 'p' passes a round of the prime testing. 'callback' will be called as follows, callback(1,n,cb_arg) where n is the number of the round, just passed. As per usual 'ctx' contains temporary variables used. If ctx is NULL, it does not matter, a local version will be malloced. This parameter is present to save some mallocing inside the function but probably could be removed. 0 is returned on error. 'ncheck' is the number of Miller-Rabin tests to run. It is suggested to use the value 'BN_prime_checks' by default. BIGNUM *BN_generate_prime( int bits, int strong, BIGNUM *a, BIGNUM *rems, void (*callback)()); char *cb_arg This function is used to generate prime numbers. It returns a new BIGNUM that has a high probability of being a prime. 'bits' is the number of bits that are to be in the prime. If 'strong' is true, the returned prime will also be a strong prime ((p-1)/2 is also prime). While searching for the prime ('p'), we can add the requirement that the prime fill the following condition p%a == rem. This can be used to help search for primes with specific features, which is required when looking for primes suitable for use with certain 'g' values in the Diffie-Hellman key exchange algorithm. If 'a' is NULL, this condition is not checked. If rem is NULL, rem is assumed to be 1. Since this search for a prime can take quite some time, if callback is not NULL, it is called in the following situations. We have a suspected prime (from a quick sieve), callback(0,sus_prime++,cb_arg). Each item to be passed to BN_is_prime(). callback(1,round++,cb_arg). Each successful 'round' in BN_is_prime(). callback(2,round,cb_arg). For each successful BN_is_prime() test. Hints ----- DSA wants 64*32 to use word mont mul, but RSA wants to use full. ==== callback.doc ======================================================== Callback functions used in SSLeay. -------------------------- The BIO library. Each BIO structure can have a callback defined against it. This callback is called 2 times for each BIO 'function'. It is passed 6 parameters. BIO_debug_callback() is an example callback which is defined in crypto/buffer/bio_cb.c and is used in apps/dgst.c This is intended mostly for debuging or to notify the application of IO. long BIO_debug_callback(BIO *bio,int cmd,char *argp,int argi,long argl, long ret); bio is the BIO being called, cmd is the type of BIO function being called. Look at the BIO_CB_* defines in buffer.h. Argp and argi are the arguments passed to BIO_read(), BIO_write, BIO_gets(), BIO_puts(). In the case of BIO_ctrl(), argl is also defined. The first time the callback is called, before the underlying function has been executed, 0 is passed as 'ret', and if the return code from the callback is not > 0, the call is aborted and the returned <= 0 value is returned. The second time the callback is called, the 'cmd' value also has BIO_CB_RETURN logically 'or'ed with it. The 'ret' value is the value returned from the actuall function call and whatever the callback returns is returned from the BIO function. BIO_set_callback(b,cb) can be used to set the callback function (b is a BIO), and BIO_set_callback_arg(b,arg) can be used to set the cb_arg argument in the BIO strucutre. This field is only intended to be used by application, primarily in the callback function since it is accessable since the BIO is passed. -------------------------- The PEM library. The pem library only really uses one type of callback, static int def_callback(char *buf, int num, int verify); which is used to return a password string if required. 'buf' is the buffer to put the string in. 'num' is the size of 'buf' and 'verify' is used to indicate that the password should be checked. This last flag is mostly used when reading a password for encryption. For all of these functions, a NULL callback will call the above mentioned default callback. This default function does not work under Windows 3.1. For other machines, it will use an application defined prompt string (EVP_set_pw_prompt(), which defines a library wide prompt string) if defined, otherwise it will use it's own PEM password prompt. It will then call EVP_read_pw_string() to get a password from the console. If your application wishes to use nice fancy windows to retrieve passwords, replace this function. The callback should return the number of bytes read into 'buf'. If the number of bytes <= 0, it is considered an error. Functions that take this callback are listed below. For the 'read' type functions, the callback will only be required if the PEM data is encrypted. For the Write functions, normally a password can be passed in 'kstr', of 'klen' bytes which will be used if the 'enc' cipher is not NULL. If 'kstr' is NULL, the callback will be used to retrieve a password. int PEM_do_header (EVP_CIPHER_INFO *cipher, unsigned char *data,long *len, int (*callback)()); char *PEM_ASN1_read_bio(char *(*d2i)(),char *name,BIO *bp,char **x,int (*cb)()); char *PEM_ASN1_read(char *(*d2i)(),char *name,FILE *fp,char **x,int (*cb)()); int PEM_ASN1_write_bio(int (*i2d)(),char *name,BIO *bp,char *x, EVP_CIPHER *enc,unsigned char *kstr,int klen,int (*callback)()); int PEM_ASN1_write(int (*i2d)(),char *name,FILE *fp,char *x, EVP_CIPHER *enc,unsigned char *kstr,int klen,int (*callback)()); STACK *PEM_X509_INFO_read(FILE *fp, STACK *sk, int (*cb)()); STACK *PEM_X509_INFO_read_bio(BIO *fp, STACK *sk, int (*cb)()); #define PEM_write_RSAPrivateKey(fp,x,enc,kstr,klen,cb) #define PEM_write_DSAPrivateKey(fp,x,enc,kstr,klen,cb) #define PEM_write_bio_RSAPrivateKey(bp,x,enc,kstr,klen,cb) #define PEM_write_bio_DSAPrivateKey(bp,x,enc,kstr,klen,cb) #define PEM_read_SSL_SESSION(fp,x,cb) #define PEM_read_X509(fp,x,cb) #define PEM_read_X509_REQ(fp,x,cb) #define PEM_read_X509_CRL(fp,x,cb) #define PEM_read_RSAPrivateKey(fp,x,cb) #define PEM_read_DSAPrivateKey(fp,x,cb) #define PEM_read_PrivateKey(fp,x,cb) #define PEM_read_PKCS7(fp,x,cb) #define PEM_read_DHparams(fp,x,cb) #define PEM_read_bio_SSL_SESSION(bp,x,cb) #define PEM_read_bio_X509(bp,x,cb) #define PEM_read_bio_X509_REQ(bp,x,cb) #define PEM_read_bio_X509_CRL(bp,x,cb) #define PEM_read_bio_RSAPrivateKey(bp,x,cb) #define PEM_read_bio_DSAPrivateKey(bp,x,cb) #define PEM_read_bio_PrivateKey(bp,x,cb) #define PEM_read_bio_PKCS7(bp,x,cb) #define PEM_read_bio_DHparams(bp,x,cb) int i2d_Netscape_RSA(RSA *a, unsigned char **pp, int (*cb)()); RSA *d2i_Netscape_RSA(RSA **a, unsigned char **pp, long length, int (*cb)()); Now you will notice that macros like #define PEM_write_X509(fp,x) \ PEM_ASN1_write((int (*)())i2d_X509,PEM_STRING_X509,fp, \ (char *)x, NULL,NULL,0,NULL) Don't do encryption normally. If you want to PEM encrypt your X509 structure, either just call PEM_ASN1_write directly or just define your own macro variant. As you can see, this macro just sets all encryption related parameters to NULL. -------------------------- The SSL library. #define SSL_set_info_callback(ssl,cb) #define SSL_CTX_set_info_callback(ctx,cb) void callback(SSL *ssl,int location,int ret) This callback is called each time around the SSL_connect()/SSL_accept() state machine. So it will be called each time the SSL protocol progresses. It is mostly present for use when debugging. When SSL_connect() or SSL_accept() return, the location flag is SSL_CB_ACCEPT_EXIT or SSL_CB_CONNECT_EXIT and 'ret' is the value about to be returned. Have a look at the SSL_CB_* defines in ssl.h. If an info callback is defined against the SSL_CTX, it is called unless there is one set against the SSL. Have a look at void client_info_callback() in apps/s_client() for an example. Certificate verification. void SSL_set_verify(SSL *s, int mode, int (*callback) ()); void SSL_CTX_set_verify(SSL_CTX *ctx,int mode,int (*callback)()); This callback is used to help verify client and server X509 certificates. It is actually passed to X509_cert_verify(), along with the SSL structure so you have to read about X509_cert_verify() :-). The SSL_CTX version is used if the SSL version is not defined. X509_cert_verify() is the function used by the SSL part of the library to verify certificates. This function is nearly always defined by the application. void SSL_CTX_set_cert_verify_cb(SSL_CTX *ctx, int (*cb)(),char *arg); int callback(char *arg,SSL *s,X509 *xs,STACK *cert_chain); This call is used to replace the SSLeay certificate verification code. The 'arg' is kept in the SSL_CTX and is passed to the callback. If the callback returns 0, the certificate is rejected, otherwise it is accepted. The callback is replacing the X509_cert_verify() call. This feature is not often used, but if you wished to implement some totally different certificate authentication system, this 'hook' is vital. SSLeay keeps a cache of session-ids against each SSL_CTX. These callbacks can be used to notify the application when a SSL_SESSION is added to the cache or to retrieve a SSL_SESSION that is not in the cache from the application. #define SSL_CTX_sess_set_get_cb(ctx,cb) SSL_SESSION *callback(SSL *s,char *session_id,int session_id_len,int *copy); If defined, this callback is called to return the SESSION_ID for the session-id in 'session_id', of 'session_id_len' bytes. 'copy' is set to 1 if the server is to 'take a copy' of the SSL_SESSION structure. It is 0 if the SSL_SESSION is being 'passed in' so the SSLeay library is now responsible for 'free()ing' the structure. Basically it is used to indicate if the reference count on the SSL_SESSION structure needs to be incremented. #define SSL_CTX_sess_set_new_cb(ctx,cb) int callback(SSL *s, SSL_SESSION *sess); When a new connection is established, if the SSL_SESSION is going to be added to the cache, this callback is called. Return 1 if a 'copy' is required, otherwise, return 0. This return value just causes the reference count to be incremented (on return of a 1), this means the application does not need to worry about incrementing the refernece count (and the locking that implies in a multi-threaded application). void SSL_CTX_set_default_passwd_cb(SSL_CTX *ctx,int (*cb)()); This sets the SSL password reading function. It is mostly used for windowing applications and used by PEM_read_bio_X509() and PEM_read_bio_RSAPrivateKey() calls inside the SSL library. The only reason this is present is because the calls to PEM_* functions is hidden in the SSLeay library so you have to pass in the callback some how. #define SSL_CTX_set_client_cert_cb(ctx,cb) int callback(SSL *s,X509 **x509, EVP_PKEY **pkey); Called when a client certificate is requested but there is not one set against the SSL_CTX or the SSL. If the callback returns 1, x509 and pkey need to point to valid data. The library will free these when required so if the application wants to keep these around, increment their reference counts. If 0 is returned, no client cert is available. If -1 is returned, it is assumed that the callback needs to be called again at a later point in time. SSL_connect will return -1 and SSL_want_x509_lookup(ssl) returns true. Remember that application data can be attached to an SSL structure via the SSL_set_app_data(SSL *ssl,char *data) call. -------------------------- The X509 library. int X509_cert_verify(CERTIFICATE_CTX *ctx,X509 *xs, int (*cb)(), int *error,char *arg,STACK *cert_chain); int verify_callback(int ok,X509 *xs,X509 *xi,int depth,int error,char *arg, STACK *cert_chain); X509_cert_verify() is used to authenticate X509 certificates. The 'ctx' holds the details of the various caches and files used to locate certificates. 'xs' is the certificate to verify and 'cb' is the application callback (more detail later). 'error' will be set to the error code and 'arg' is passed to the 'cb' callback. Look at the VERIFY_* defines in crypto/x509/x509.h When ever X509_cert_verify() makes a 'negative' decision about a certitificate, the callback is called. If everything checks out, the callback is called with 'VERIFY_OK' or 'VERIFY_ROOT_OK' (for a self signed cert that is not the passed certificate). The callback is passed the X509_cert_verify opinion of the certificate in 'ok', the certificate in 'xs', the issuer certificate in 'xi', the 'depth' of the certificate in the verification 'chain', the VERIFY_* code in 'error' and the argument passed to X509_cert_verify() in 'arg'. cert_chain is a list of extra certs to use if they are not in the cache. The callback can be used to look at the error reason, and then return 0 for an 'error' or '1' for ok. This will override the X509_cert_verify() opinion of the certificates validity. Processing will continue depending on the return value. If one just wishes to use the callback for informational reason, just return the 'ok' parameter. -------------------------- The BN and DH library. BIGNUM *BN_generate_prime(int bits,int strong,BIGNUM *add, BIGNUM *rem,void (*callback)(int,int)); int BN_is_prime(BIGNUM *p,int nchecks,void (*callback)(int,int), Read doc/bn.doc for the description of these 2. DH *DH_generate_parameters(int prime_len,int generator, void (*callback)(int,int)); Read doc/bn.doc for the description of the callback, since it is just passed to BN_generate_prime(), except that it is also called as callback(3,0) by this function. -------------------------- The CRYPTO library. void CRYPTO_set_locking_callback(void (*func)(int mode,int type,char *file, int line)); void CRYPTO_set_add_lock_callback(int (*func)(int *num,int mount, int type,char *file, int line)); void CRYPTO_set_id_callback(unsigned long (*func)(void)); Read threads.doc for info on these ones. ==== cipher.doc ======================================================== The Cipher subroutines. These routines require "evp.h" to be included. These functions are a higher level interface to the various cipher routines found in this library. As such, they allow the same code to be used to encrypt and decrypt via different ciphers with only a change in an initial parameter. These routines also provide buffering for block ciphers. These routines all take a pointer to the following structure to specify which cipher to use. If you wish to use a new cipher with these routines, you would probably be best off looking an how an existing cipher is implemented and copying it. At this point in time, I'm not going to go into many details. This structure should be considered opaque typedef struct pem_cipher_st { int type; int block_size; int key_len; int iv_len; void (*enc_init)(); /* init for encryption */ void (*dec_init)(); /* init for decryption */ void (*do_cipher)(); /* encrypt data */ } EVP_CIPHER; The type field is the object NID of the cipher type (read the section on Objects for an explanation of what a NID is). The cipher block_size is how many bytes need to be passed to the cipher at a time. Key_len is the length of the key the cipher requires and iv_len is the length of the initialisation vector required. enc_init is the function called to initialise the ciphers context for encryption and dec_init is the function to initialise for decryption (they need to be different, especially for the IDEA cipher). One reason for specifying the Cipher via a pointer to a structure is that if you only use des-cbc, only the des-cbc routines will be included when you link the program. If you passed an integer that specified which cipher to use, the routine that mapped that integer to a set of cipher functions would cause all the ciphers to be link into the code. This setup also allows new ciphers to be added by the application (with some restrictions). The thirteen ciphers currently defined in this library are EVP_CIPHER *EVP_des_ecb(); /* DES in ecb mode, iv=0, block=8, key= 8 */ EVP_CIPHER *EVP_des_ede(); /* DES in ecb ede mode, iv=0, block=8, key=16 */ EVP_CIPHER *EVP_des_ede3(); /* DES in ecb ede mode, iv=0, block=8, key=24 */ EVP_CIPHER *EVP_des_cfb(); /* DES in cfb mode, iv=8, block=1, key= 8 */ EVP_CIPHER *EVP_des_ede_cfb(); /* DES in ede cfb mode, iv=8, block=1, key=16 */ EVP_CIPHER *EVP_des_ede3_cfb();/* DES in ede cfb mode, iv=8, block=1, key=24 */ EVP_CIPHER *EVP_des_ofb(); /* DES in ofb mode, iv=8, block=1, key= 8 */ EVP_CIPHER *EVP_des_ede_ofb(); /* DES in ede ofb mode, iv=8, block=1, key=16 */ EVP_CIPHER *EVP_des_ede3_ofb();/* DES in ede ofb mode, iv=8, block=1, key=24 */ EVP_CIPHER *EVP_des_cbc(); /* DES in cbc mode, iv=8, block=8, key= 8 */ EVP_CIPHER *EVP_des_ede_cbc(); /* DES in cbc ede mode, iv=8, block=8, key=16 */ EVP_CIPHER *EVP_des_ede3_cbc();/* DES in cbc ede mode, iv=8, block=8, key=24 */ EVP_CIPHER *EVP_desx_cbc(); /* DES in desx cbc mode,iv=8, block=8, key=24 */ EVP_CIPHER *EVP_rc4(); /* RC4, iv=0, block=1, key=16 */ EVP_CIPHER *EVP_idea_ecb(); /* IDEA in ecb mode, iv=0, block=8, key=16 */ EVP_CIPHER *EVP_idea_cfb(); /* IDEA in cfb mode, iv=8, block=1, key=16 */ EVP_CIPHER *EVP_idea_ofb(); /* IDEA in ofb mode, iv=8, block=1, key=16 */ EVP_CIPHER *EVP_idea_cbc(); /* IDEA in cbc mode, iv=8, block=8, key=16 */ EVP_CIPHER *EVP_rc2_ecb(); /* RC2 in ecb mode, iv=0, block=8, key=16 */ EVP_CIPHER *EVP_rc2_cfb(); /* RC2 in cfb mode, iv=8, block=1, key=16 */ EVP_CIPHER *EVP_rc2_ofb(); /* RC2 in ofb mode, iv=8, block=1, key=16 */ EVP_CIPHER *EVP_rc2_cbc(); /* RC2 in cbc mode, iv=8, block=8, key=16 */ EVP_CIPHER *EVP_bf_ecb(); /* Blowfish in ecb mode,iv=0, block=8, key=16 */ EVP_CIPHER *EVP_bf_cfb(); /* Blowfish in cfb mode,iv=8, block=1, key=16 */ EVP_CIPHER *EVP_bf_ofb(); /* Blowfish in ofb mode,iv=8, block=1, key=16 */ EVP_CIPHER *EVP_bf_cbc(); /* Blowfish in cbc mode,iv=8, block=8, key=16 */ The meaning of the compound names is as follows. des The base cipher is DES. idea The base cipher is IDEA rc4 The base cipher is RC4-128 rc2 The base cipher is RC2-128 ecb Electronic Code Book form of the cipher. cbc Cipher Block Chaining form of the cipher. cfb 64 bit Cipher Feedback form of the cipher. ofb 64 bit Output Feedback form of the cipher. ede The cipher is used in Encrypt, Decrypt, Encrypt mode. The first and last keys are the same. ede3 The cipher is used in Encrypt, Decrypt, Encrypt mode. All the Cipher routines take a EVP_CIPHER_CTX pointer as an argument. The state of the cipher is kept in this structure. typedef struct EVP_CIPHER_Ctx_st { EVP_CIPHER *cipher; int encrypt; /* encrypt or decrypt */ int buf_len; /* number we have left */ unsigned char buf[8]; union { .... /* cipher specific stuff */ } c; } EVP_CIPHER_CTX; Cipher is a pointer the the EVP_CIPHER for the current context. The encrypt flag indicates encryption or decryption. buf_len is the number of bytes currently being held in buf. The 'c' union holds the cipher specify context. The following functions are to be used. int EVP_read_pw_string( char *buf, int len, char *prompt, int verify, This function is the same as des_read_pw_string() (des.doc). void EVP_set_pw_prompt(char *prompt); This function sets the 'default' prompt to use to use in EVP_read_pw_string when the prompt parameter is NULL. If the prompt parameter is NULL, this 'default prompt' feature is turned off. Be warned, this is a global variable so weird things will happen if it is used under Win16 and care must be taken with a multi-threaded version of the library. char *EVP_get_pw_prompt(); This returns a pointer to the default prompt string. NULL if it is not set. int EVP_BytesToKey( EVP_CIPHER *type, EVP_MD *md, unsigned char *salt, unsigned char *data, int datal, int count, unsigned char *key, unsigned char *iv); This function is used to generate a key and an initialisation vector for a specified cipher from a key string and a salt. Type specifies the cipher the 'key' is being generated for. Md is the message digest algorithm to use to generate the key and iv. The salt is an optional 8 byte object that is used to help seed the key generator. If the salt value is NULL, it is just not used. Datal is the number of bytes to use from 'data' in the key generation. This function returns the key size for the specified cipher, if data is NULL, this value is returns and no other computation is performed. Count is the number of times to loop around the key generator. I would suggest leaving it's value as 1. Key and iv are the structures to place the returning iv and key in. If they are NULL, no value is generated for that particular value. The algorithm used is as follows /* M[] is an array of message digests * MD() is the message digest function */ M[0]=MD(data . salt); for (i=1; i<count; i++) M[0]=MD(M[0]); i=1 while (data still needed for key and iv) { M[i]=MD(M[i-1] . data . salt); for (i=1; i<count; i++) M[i]=MD(M[i]); i++; } If the salt is NULL, it is not used. The digests are concatenated together. M = M[0] . M[1] . M[2] ....... For key= 8, iv=8 => key=M[0.. 8], iv=M[ 9 .. 16]. For key=16, iv=0 => key=M[0..16]. For key=16, iv=8 => key=M[0..16], iv=M[17 .. 24]. For key=24, iv=8 => key=M[0..24], iv=M[25 .. 32]. This routine will produce DES-CBC keys and iv that are compatible with the PKCS-5 standard when md2 or md5 are used. If md5 is used, the salt is NULL and count is 1, this routine will produce the password to key mapping normally used with RC4. I have attempted to logically extend the PKCS-5 standard to generate keys and iv for ciphers that require more than 16 bytes, if anyone knows what the correct standard is, please inform me. When using sha or sha1, things are a bit different under this scheme, since sha produces a 20 byte digest. So for ciphers requiring 24 bits of data, 20 will come from the first MD and 4 will come from the second. I have considered having a separate function so this 'routine' can be used without the requirement of passing a EVP_CIPHER *, but I have decided to not bother. If you wish to use the function without official EVP_CIPHER structures, just declare a local one and set the key_len and iv_len fields to the length you desire. The following routines perform encryption and decryption 'by parts'. By this I mean that there are groups of 3 routines. An Init function that is used to specify a cipher and initialise data structures. An Update routine that does encryption/decryption, one 'chunk' at a time. And finally a 'Final' function that finishes the encryption/decryption process. All these functions take a EVP_CIPHER pointer to specify which cipher to encrypt/decrypt with. They also take a EVP_CIPHER_CTX object as an argument. This structure is used to hold the state information associated with the operation in progress. void EVP_EncryptInit( EVP_CIPHER_CTX *ctx, EVP_CIPHER *type, unsigned char *key, unsigned char *iv); This function initialise a EVP_CIPHER_CTX for encryption using the cipher passed in the 'type' field. The cipher is initialised to use 'key' as the key and 'iv' for the initialisation vector (if one is required). If the type, key or iv is NULL, the value currently in the EVP_CIPHER_CTX is reused. So to perform several decrypt using the same cipher, key and iv, initialise with the cipher, key and iv the first time and then for subsequent calls, reuse 'ctx' but pass NULL for type, key and iv. You must make sure to pass a key that is large enough for a particular cipher. I would suggest using the EVP_BytesToKey() function. void EVP_EncryptUpdate( EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl, unsigned char *in, int inl); This function takes 'inl' bytes from 'in' and outputs bytes encrypted by the cipher 'ctx' was initialised with into 'out'. The number of bytes written to 'out' is put into outl. If a particular cipher encrypts in blocks, less or more bytes than input may be output. Currently the largest block size used by supported ciphers is 8 bytes, so 'out' should have room for 'inl+7' bytes. Normally EVP_EncryptInit() is called once, followed by lots and lots of calls to EVP_EncryptUpdate, followed by a single EVP_EncryptFinal call. void EVP_EncryptFinal( EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl); Because quite a large number of ciphers are block ciphers, there is often an incomplete block to write out at the end of the encryption. EVP_EncryptFinal() performs processing on this last block. The last block in encoded in such a way that it is possible to determine how many bytes in the last block are valid. For 8 byte block size ciphers, if only 5 bytes in the last block are valid, the last three bytes will be filled with the value 3. If only 2 were valid, the other 6 would be filled with sixes. If all 8 bytes are valid, a extra 8 bytes are appended to the cipher stream containing nothing but 8 eights. These last bytes are output into 'out' and the number of bytes written is put into 'outl' These last bytes are output into 'out' and the number of bytes written is put into 'outl'. This form of block cipher finalisation is compatible with PKCS-5. Please remember that even if you are using ciphers like RC4 that has no blocking and so the function will not write anything into 'out', it would still be a good idea to pass a variable for 'out' that can hold 8 bytes just in case the cipher is changed some time in the future. It should also be remembered that the EVP_CIPHER_CTX contains the password and so when one has finished encryption with a particular EVP_CIPHER_CTX, it is good practice to zero the structure (ie. memset(ctx,0,sizeof(EVP_CIPHER_CTX)). void EVP_DecryptInit( EVP_CIPHER_CTX *ctx, EVP_CIPHER *type, unsigned char *key, unsigned char *iv); This function is basically the same as EVP_EncryptInit() accept that is prepares the EVP_CIPHER_CTX for decryption. void EVP_DecryptUpdate( EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl, unsigned char *in, int inl); This function is basically the same as EVP_EncryptUpdate() except that it performs decryption. There is one fundamental difference though. 'out' can not be the same as 'in' for any ciphers with a block size greater than 1 if more than one call to EVP_DecryptUpdate() will be made. This is because this routine can hold a 'partial' block between calls. When a partial block is decrypted (due to more bytes being passed via this function, they will be written to 'out' overwriting the input bytes in 'in' that have not been read yet. From this it should also be noted that 'out' should be at least one 'block size' larger than 'inl'. This problem only occurs on the second and subsequent call to EVP_DecryptUpdate() when using a block cipher. int EVP_DecryptFinal( EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl); This function is different to EVP_EncryptFinal in that it 'removes' any padding bytes appended when the data was encrypted. Due to the way in which 1 to 8 bytes may have been appended when encryption using a block cipher, 'out' can end up with 0 to 7 bytes being put into it. When decoding the padding bytes, it is possible to detect an incorrect decryption. If the decryption appears to be wrong, 0 is returned. If everything seems ok, 1 is returned. For ciphers with a block size of 1 (RC4), this function would normally not return any bytes and would always return 1. Just because this function returns 1 does not mean the decryption was correct. It would normally be wrong due to either the wrong key/iv or corruption of the cipher data fed to EVP_DecryptUpdate(). As for EVP_EncryptFinal, it is a good idea to zero the EVP_CIPHER_CTX after use since the structure contains the key used to decrypt the data. The following Cipher routines are convenience routines that call either EVP_EncryptXxx or EVP_DecryptXxx depending on weather the EVP_CIPHER_CTX was setup to encrypt or decrypt. void EVP_CipherInit( EVP_CIPHER_CTX *ctx, EVP_CIPHER *type, unsigned char *key, unsigned char *iv, int enc); This function take arguments that are the same as EVP_EncryptInit() and EVP_DecryptInit() except for the extra 'enc' flag. If 1, the EVP_CIPHER_CTX is setup for encryption, if 0, decryption. void EVP_CipherUpdate( EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl, unsigned char *in, int inl); Again this function calls either EVP_EncryptUpdate() or EVP_DecryptUpdate() depending on state in the 'ctx' structure. As noted for EVP_DecryptUpdate(), when this routine is used for decryption with block ciphers, 'out' should not be the same as 'in'. int EVP_CipherFinal( EVP_CIPHER_CTX *ctx, unsigned char *outm, int *outl); This routine call EVP_EncryptFinal() or EVP_DecryptFinal() depending on the state information in 'ctx'. 1 is always returned if the mode is encryption, otherwise the return value is the return value of EVP_DecryptFinal(). ==== cipher.m ======================================================== Date: Tue, 15 Oct 1996 08:16:14 +1000 (EST) From: Eric Young <eay@mincom.com> X-Sender: eay@orb To: Roland Haring <rharing@tandem.cl> Cc: ssl-users@mincom.com Subject: Re: Symmetric encryption with ssleay In-Reply-To: <m0vBpyq-00001aC@tandemnet.tandem.cl> Message-Id: <Pine.SOL.3.91.961015075623.11394A-100000@orb> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: ssl-lists-owner@mincom.com Precedence: bulk Status: RO X-Status: On Fri, 11 Oct 1996, Roland Haring wrote: > THE_POINT: > Would somebody be so kind to give me the minimum basic > calls I need to do to libcrypto.a to get some text encrypted > and decrypted again? ...hopefully with code included to do > base64 encryption and decryption ... e.g. that sign-it.c code > posted some while ago was a big help :-) (please, do not point > me to apps/enc.c where I suspect my Heissenbug to be hidden :-) Ok, the base64 encoding stuff in 'enc.c' does the wrong thing sometimes when the data is less than a line long (this is for decoding). I'll dig up the exact fix today and post it. I am taking longer on 0.6.5 than I intended so I'll just post this patch. The documentation to read is in doc/cipher.doc, doc/encode.doc (very sparse :-). and perhaps doc/digest.doc, The basic calls to encrypt with say triple DES are Given char key[EVP_MAX_KEY_LENGTH]; char iv[EVP_MAX_IV_LENGTH]; EVP_CIPHER_CTX ctx; unsigned char out[512+8]; int outl; /* optional generation of key/iv data from text password using md5 * via an upward compatable verson of PKCS#5. */ EVP_BytesToKey(EVP_des_ede3_cbc,EVP_md5,NULL,passwd,strlen(passwd), key,iv); /* Initalise the EVP_CIPHER_CTX */ EVP_EncryptInit(ctx,EVP_des_ede3_cbc,key,iv); while (....) { /* This is processing 512 bytes at a time, the bytes are being * copied into 'out', outl bytes are output. 'out' should not be the * same as 'in' for reasons mentioned in the documentation. */ EVP_EncryptUpdate(ctx,out,&outl,in,512); } /* Output the last 'block'. If the cipher is a block cipher, the last * block is encoded in such a way so that a wrong decryption will normally be * detected - again, one of the PKCS standards. */ EVP_EncryptFinal(ctx,out,&outl); To decrypt, use the EVP_DecryptXXXXX functions except that EVP_DecryptFinal() will return 0 if the decryption fails (only detectable on block ciphers). You can also use EVP_CipherInit() EVP_CipherUpdate() EVP_CipherFinal() which does either encryption or decryption depending on an extra parameter to EVP_CipherInit(). To do the base64 encoding, EVP_EncodeInit() EVP_EncodeUpdate() EVP_EncodeFinal() EVP_DecodeInit() EVP_DecodeUpdate() EVP_DecodeFinal() where the encoding is quite simple, but the decoding can be a bit more fun (due to dud input). EVP_DecodeUpdate() returns -1 for an error on an input line, 0 if the 'last line' was just processed, and 1 if more lines should be submitted. EVP_DecodeFinal() returns -1 for an error or 1 if things are ok. So the loop becomes EVP_DecodeInit(....) for (;;) { i=EVP_DecodeUpdate(....); if (i < 0) goto err; /* process the data */ if (i == 0) break; } EVP_DecodeFinal(....); /* process the data */ The problem in 'enc.c' is that I was stuff the processing up after the EVP_DecodeFinal(...) when the for(..) loop was not being run (one line of base64 data) and this was because 'enc.c' tries to scan over a file until it hits the first valid base64 encoded line. hope this helps a bit. eric -- Eric Young | BOOL is tri-state according to Bill Gates. AARNet: eay@mincom.oz.au | RTFM Win32 GetMessage(). ==== conf.doc ======================================================== The CONF library. The CONF library is a simple set of routines that can be used to configure programs. It is a superset of the genenv() function with some extra structure. The library consists of 5 functions. LHASH *CONF_load(LHASH *config,char *file); This function is called to load in a configuration file. Multiple configuration files can be loaded, with each subsequent 'load' overwriting any already defined 'variables'. If there is an error, NULL is returned. If config is NULL, a new LHASH structure is created and returned, otherwise the new data in the 'file' is loaded into the 'config' structure. void CONF_free(LHASH *config); This function free()s the data in config. char *CONF_get_string(LHASH *config,char *section,char *name); This function returns the string found in 'config' that corresponds to the 'section' and 'name' specified. Classes and the naming system used will be discussed later in this document. If the variable is not defined, an NULL is returned. long CONF_get_long(LHASH *config,char *section, char *name); This function is the same as CONF_get_string() except that it converts the string to an long and returns it. If variable is not a number or the variable does not exist, 0 is returned. This is a little problematic but I don't know of a simple way around it. STACK *CONF_get_section(LHASH *config, char *section); This function returns a 'stack' of CONF_VALUE items that are all the items defined in a particular section. DO NOT free() any of the variable returned. They will disappear when CONF_free() is called. The 'lookup' model. The configuration file is divided into 'sections'. Each section is started by a line of the form '[ section ]'. All subsequent variable definitions are of this section. A variable definition is a simple alpha-numeric name followed by an '=' and then the data. A section or variable name can be described by a regular expression of the following form '[A-Za-z0-9_]+'. The value of the variable is the text after the '=' until the end of the line, stripped of leading and trailing white space. At this point I should mention that a '#' is a comment character, \ is the escape character, and all three types of quote can be used to stop any special interpretation of the data. Now when the data is being loaded, variable expansion can occur. This is done by expanding any $NAME sequences into the value represented by the variable NAME. If the variable is not in the current section, the different section can be specified by using the $SECTION::NAME form. The ${NAME} form also works and is very useful for expanding variables inside strings. When a variable is looked up, there are 2 special section. 'default', which is the initial section, and 'ENV' which is the processes environment variables (accessed via getenv()). When a variable is looked up, it is first 'matched' with it's section (if one was specified), if this fails, the 'default' section is matched. If the 'lhash' variable passed was NULL, the environment is searched. Now why do we bother with sections? So we can have multiple programs using the same configuration file, or multiple instances of the same program using different variables. It also provides a nice mechanism to override the processes environment variables (eg ENV::HOME=/tmp). If there is a program specific variable missing, we can have default values. Multiple configuration files can be loaded, with each new value clearing any predefined values. A system config file can provide 'default' values, and application/usr specific files can provide overriding values. Examples # This is a simple example SSLEAY_HOME = /usr/local/ssl ENV::PATH = $SSLEAY_HOME/bin:$PATH # override my path [X509] cert_dir = $SSLEAY_HOME/certs # /usr/local/ssl/certs [SSL] CIPHER = DES-EDE-MD5:RC4-MD5 USER_CERT = $HOME/${USER}di'r 5' # /home/eay/eaydir 5 USER_CERT = $HOME/\${USER}di\'r # /home/eay/${USER}di'r USER_CERT = "$HOME/${US"ER}di\'r # $HOME/${USER}di'r TEST = 1234\ 5678\ 9ab # TEST=123456789ab TTT = 1234\n\n # TTT=1234<nl><nl> ==== des.doc ======================================================== The DES library. Please note that this library was originally written to operate with eBones, a version of Kerberos that had had encryption removed when it left the USA and then put back in. As such there are some routines that I will advise not using but they are still in the library for historical reasons. For all calls that have an 'input' and 'output' variables, they can be the same. This library requires the inclusion of 'des.h'. All of the encryption functions take what is called a des_key_schedule as an argument. A des_key_schedule is an expanded form of the des key. A des_key is 8 bytes of odd parity, the type used to hold the key is a des_cblock. A des_cblock is an array of 8 bytes, often in this library description I will refer to input bytes when the function specifies des_cblock's as input or output, this just means that the variable should be a multiple of 8 bytes. The define DES_ENCRYPT is passed to specify encryption, DES_DECRYPT to specify decryption. The functions and global variable are as follows: int des_check_key; DES keys are supposed to be odd parity. If this variable is set to a non-zero value, des_set_key() will check that the key has odd parity and is not one of the known weak DES keys. By default this variable is turned off; void des_set_odd_parity( des_cblock *key ); This function takes a DES key (8 bytes) and sets the parity to odd. int des_is_weak_key( des_cblock *key ); This function returns a non-zero value if the DES key passed is a weak, DES key. If it is a weak key, don't use it, try a different one. If you are using 'random' keys, the chances of hitting a weak key are 1/2^52 so it is probably not worth checking for them. int des_set_key( des_cblock *key, des_key_schedule schedule); Des_set_key converts an 8 byte DES key into a des_key_schedule. A des_key_schedule is an expanded form of the key which is used to perform actual encryption. It can be regenerated from the DES key so it only needs to be kept when encryption or decryption is about to occur. Don't save or pass around des_key_schedule's since they are CPU architecture dependent, DES keys are not. If des_check_key is non zero, zero is returned if the key has the wrong parity or the key is a weak key, else 1 is returned. int des_key_sched( des_cblock *key, des_key_schedule schedule); An alternative name for des_set_key(). int des_rw_mode; /* defaults to DES_PCBC_MODE */ This flag holds either DES_CBC_MODE or DES_PCBC_MODE (default). This specifies the function to use in the enc_read() and enc_write() functions. void des_encrypt( unsigned long *data, des_key_schedule ks, int enc); This is the DES encryption function that gets called by just about every other DES routine in the library. You should not use this function except to implement 'modes' of DES. I say this because the functions that call this routine do the conversion from 'char *' to long, and this needs to be done to make sure 'non-aligned' memory access do not occur. The characters are loaded 'little endian', have a look at my source code for more details on how I use this function. Data is a pointer to 2 unsigned long's and ks is the des_key_schedule to use. enc, is non zero specifies encryption, zero if decryption. void des_encrypt2( unsigned long *data, des_key_schedule ks, int enc); This functions is the same as des_encrypt() except that the DES initial permutation (IP) and final permutation (FP) have been left out. As for des_encrypt(), you should not use this function. It is used by the routines in my library that implement triple DES. IP() des_encrypt2() des_encrypt2() des_encrypt2() FP() is the same as des_encrypt() des_encrypt() des_encrypt() except faster :-). void des_ecb_encrypt( des_cblock *input, des_cblock *output, des_key_schedule ks, int enc); This is the basic Electronic Code Book form of DES, the most basic form. Input is encrypted into output using the key represented by ks. If enc is non zero (DES_ENCRYPT), encryption occurs, otherwise decryption occurs. Input is 8 bytes long and output is 8 bytes. (the des_cblock structure is 8 chars). void des_ecb3_encrypt( des_cblock *input, des_cblock *output, des_key_schedule ks1, des_key_schedule ks2, des_key_schedule ks3, int enc); This is the 3 key EDE mode of ECB DES. What this means is that the 8 bytes of input is encrypted with ks1, decrypted with ks2 and then encrypted again with ks3, before being put into output; C=E(ks3,D(ks2,E(ks1,M))). There is a macro, des_ecb2_encrypt() that only takes 2 des_key_schedules that implements, C=E(ks1,D(ks2,E(ks1,M))) in that the final encrypt is done with ks1. void des_cbc_encrypt( des_cblock *input, des_cblock *output, long length, des_key_schedule ks, des_cblock *ivec, int enc); This routine implements DES in Cipher Block Chaining mode. Input, which should be a multiple of 8 bytes is encrypted (or decrypted) to output which will also be a multiple of 8 bytes. The number of bytes is in length (and from what I've said above, should be a multiple of 8). If length is not a multiple of 8, I'm not being held responsible :-). ivec is the initialisation vector. This function does not modify this variable. To correctly implement cbc mode, you need to do one of 2 things; copy the last 8 bytes of cipher text for use as the next ivec in your application, or use des_ncbc_encrypt(). Only this routine has this problem with updating the ivec, all other routines that are implementing cbc mode update ivec. void des_ncbc_encrypt( des_cblock *input, des_cblock *output, long length, des_key_schedule sk, des_cblock *ivec, int enc); For historical reasons, des_cbc_encrypt() did not update the ivec with the value requires so that subsequent calls to des_cbc_encrypt() would 'chain'. This was needed so that the same 'length' values would not need to be used when decrypting. des_ncbc_encrypt() does the right thing. It is the same as des_cbc_encrypt accept that ivec is updates with the correct value to pass in subsequent calls to des_ncbc_encrypt(). I advise using des_ncbc_encrypt() instead of des_cbc_encrypt(); void des_xcbc_encrypt( des_cblock *input, des_cblock *output, long length, des_key_schedule sk, des_cblock *ivec, des_cblock *inw, des_cblock *outw, int enc); This is RSA's DESX mode of DES. It uses inw and outw to 'whiten' the encryption. inw and outw are secret (unlike the iv) and are as such, part of the key. So the key is sort of 24 bytes. This is much better than cbc des. void des_3cbc_encrypt( des_cblock *input, des_cblock *output, long length, des_key_schedule sk1, des_key_schedule sk2, des_cblock *ivec1, des_cblock *ivec2, int enc); This function is flawed, do not use it. I have left it in the library because it is used in my des(1) program and will function correctly when used by des(1). If I removed the function, people could end up unable to decrypt files. This routine implements outer triple cbc encryption using 2 ks and 2 ivec's. Use des_ede2_cbc_encrypt() instead. void des_ede3_cbc_encrypt( des_cblock *input, des_cblock *output, long length, des_key_schedule ks1, des_key_schedule ks2, des_key_schedule ks3, des_cblock *ivec, int enc); This function implements outer triple CBC DES encryption with 3 keys. What this means is that each 'DES' operation inside the cbc mode is really an C=E(ks3,D(ks2,E(ks1,M))). Again, this is cbc mode so an ivec is requires. This mode is used by SSL. There is also a des_ede2_cbc_encrypt() that only uses 2 des_key_schedule's, the first being reused for the final encryption. C=E(ks1,D(ks2,E(ks1,M))). This form of triple DES is used by the RSAref library. void des_pcbc_encrypt( des_cblock *input, des_cblock *output, long length, des_key_schedule ks, des_cblock *ivec, int enc); This is Propagating Cipher Block Chaining mode of DES. It is used by Kerberos v4. It's parameters are the same as des_ncbc_encrypt(). void des_cfb_encrypt( unsigned char *in, unsigned char *out, int numbits, long length, des_key_schedule ks, des_cblock *ivec, int enc); Cipher Feedback Back mode of DES. This implementation 'feeds back' in numbit blocks. The input (and output) is in multiples of numbits bits. numbits should to be a multiple of 8 bits. Length is the number of bytes input. If numbits is not a multiple of 8 bits, the extra bits in the bytes will be considered padding. So if numbits is 12, for each 2 input bytes, the 4 high bits of the second byte will be ignored. So to encode 72 bits when using a numbits of 12 take 12 bytes. To encode 72 bits when using numbits of 9 will take 16 bytes. To encode 80 bits when using numbits of 16 will take 10 bytes. etc, etc. This padding will apply to both input and output. void des_cfb64_encrypt( unsigned char *in, unsigned char *out, long length, des_key_schedule ks, des_cblock *ivec, int *num, int enc); This is one of the more useful functions in this DES library, it implements CFB mode of DES with 64bit feedback. Why is this useful you ask? Because this routine will allow you to encrypt an arbitrary number of bytes, no 8 byte padding. Each call to this routine will encrypt the input bytes to output and then update ivec and num. num contains 'how far' we are though ivec. If this does not make much sense, read more about cfb mode of DES :-). void des_ede3_cfb64_encrypt( unsigned char *in, unsigned char *out, long length, des_key_schedule ks1, des_key_schedule ks2, des_key_schedule ks3, des_cblock *ivec, int *num, int enc); Same as des_cfb64_encrypt() accept that the DES operation is triple DES. As usual, there is a macro for des_ede2_cfb64_encrypt() which reuses ks1. void des_ofb_encrypt( unsigned char *in, unsigned char *out, int numbits, long length, des_key_schedule ks, des_cblock *ivec); This is a implementation of Output Feed Back mode of DES. It is the same as des_cfb_encrypt() in that numbits is the size of the units dealt with during input and output (in bits). void des_ofb64_encrypt( unsigned char *in, unsigned char *out, long length, des_key_schedule ks, des_cblock *ivec, int *num); The same as des_cfb64_encrypt() except that it is Output Feed Back mode. void des_ede3_ofb64_encrypt( unsigned char *in, unsigned char *out, long length, des_key_schedule ks1, des_key_schedule ks2, des_key_schedule ks3, des_cblock *ivec, int *num); Same as des_ofb64_encrypt() accept that the DES operation is triple DES. As usual, there is a macro for des_ede2_ofb64_encrypt() which reuses ks1. int des_read_pw_string( char *buf, int length, char *prompt, int verify); This routine is used to get a password from the terminal with echo turned off. Buf is where the string will end up and length is the size of buf. Prompt is a string presented to the 'user' and if verify is set, the key is asked for twice and unless the 2 copies match, an error is returned. A return code of -1 indicates a system error, 1 failure due to use interaction, and 0 is success. unsigned long des_cbc_cksum( des_cblock *input, des_cblock *output, long length, des_key_schedule ks, des_cblock *ivec); This function produces an 8 byte checksum from input that it puts in output and returns the last 4 bytes as a long. The checksum is generated via cbc mode of DES in which only the last 8 byes are kept. I would recommend not using this function but instead using the EVP_Digest routines, or at least using MD5 or SHA. This function is used by Kerberos v4 so that is why it stays in the library. char *des_fcrypt( const char *buf, const char *salt char *ret); This is my fast version of the unix crypt(3) function. This version takes only a small amount of space relative to other fast crypt() implementations. This is different to the normal crypt in that the third parameter is the buffer that the return value is written into. It needs to be at least 14 bytes long. This function is thread safe, unlike the normal crypt. char *crypt( const char *buf, const char *salt); This function calls des_fcrypt() with a static array passed as the third parameter. This emulates the normal non-thread safe semantics of crypt(3). void des_string_to_key( char *str, des_cblock *key); This function takes str and converts it into a DES key. I would recommend using MD5 instead and use the first 8 bytes of output. When I wrote the first version of these routines back in 1990, MD5 did not exist but I feel these routines are still sound. This routines is compatible with the one in MIT's libdes. void des_string_to_2keys( char *str, des_cblock *key1, des_cblock *key2); This function takes str and converts it into 2 DES keys. I would recommend using MD5 and using the 16 bytes as the 2 keys. I have nothing against these 2 'string_to_key' routines, it's just that if you say that your encryption key is generated by using the 16 bytes of an MD5 hash, every-one knows how you generated your keys. int des_read_password( des_cblock *key, char *prompt, int verify); This routine combines des_read_pw_string() with des_string_to_key(). int des_read_2passwords( des_cblock *key1, des_cblock *key2, char *prompt, int verify); This routine combines des_read_pw_string() with des_string_to_2key(). void des_random_seed( des_cblock key); This routine sets a starting point for des_random_key(). void des_random_key( des_cblock ret); This function return a random key. Make sure to 'seed' the random number generator (with des_random_seed()) before using this function. I personally now use a MD5 based random number system. int des_enc_read( int fd, char *buf, int len, des_key_schedule ks, des_cblock *iv); This function will write to a file descriptor the encrypted data from buf. This data will be preceded by a 4 byte 'byte count' and will be padded out to 8 bytes. The encryption is either CBC of PCBC depending on the value of des_rw_mode. If it is DES_PCBC_MODE, pcbc is used, if DES_CBC_MODE, cbc is used. The default is to use DES_PCBC_MODE. int des_enc_write( int fd, char *buf, int len, des_key_schedule ks, des_cblock *iv); This routines read stuff written by des_enc_read() and decrypts it. I have used these routines quite a lot but I don't believe they are suitable for non-blocking io. If you are after a full authentication/encryption over networks, have a look at SSL instead. unsigned long des_quad_cksum( des_cblock *input, des_cblock *output, long length, int out_count, des_cblock *seed); This is a function from Kerberos v4 that is not anything to do with DES but was needed. It is a cksum that is quicker to generate than des_cbc_cksum(); I personally would use MD5 routines now. ===== Modes of DES Quite a bit of the following information has been taken from AS 2805.5.2 Australian Standard Electronic funds transfer - Requirements for interfaces, Part 5.2: Modes of operation for an n-bit block cipher algorithm Appendix A There are several different modes in which DES can be used, they are as follows. Electronic Codebook Mode (ECB) (des_ecb_encrypt()) - 64 bits are enciphered at a time. - The order of the blocks can be rearranged without detection. - The same plaintext block always produces the same ciphertext block (for the same key) making it vulnerable to a 'dictionary attack'. - An error will only affect one ciphertext block. Cipher Block Chaining Mode (CBC) (des_cbc_encrypt()) - a multiple of 64 bits are enciphered at a time. - The CBC mode produces the same ciphertext whenever the same plaintext is encrypted using the same key and starting variable. - The chaining operation makes the ciphertext blocks dependent on the current and all preceding plaintext blocks and therefore blocks can not be rearranged. - The use of different starting variables prevents the same plaintext enciphering to the same ciphertext. - An error will affect the current and the following ciphertext blocks. Cipher Feedback Mode (CFB) (des_cfb_encrypt()) - a number of bits (j) <= 64 are enciphered at a time. - The CFB mode produces the same ciphertext whenever the same plaintext is encrypted using the same key and starting variable. - The chaining operation makes the ciphertext variables dependent on the current and all preceding variables and therefore j-bit variables are chained together and can not be rearranged. - The use of different starting variables prevents the same plaintext enciphering to the same ciphertext. - The strength of the CFB mode depends on the size of k (maximal if j == k). In my implementation this is always the case. - Selection of a small value for j will require more cycles through the encipherment algorithm per unit of plaintext and thus cause greater processing overheads. - Only multiples of j bits can be enciphered. - An error will affect the current and the following ciphertext variables. Output Feedback Mode (OFB) (des_ofb_encrypt()) - a number of bits (j) <= 64 are enciphered at a time. - The OFB mode produces the same ciphertext whenever the same plaintext enciphered using the same key and starting variable. More over, in the OFB mode the same key stream is produced when the same key and start variable are used. Consequently, for security reasons a specific start variable should be used only once for a given key. - The absence of chaining makes the OFB more vulnerable to specific attacks. - The use of different start variables values prevents the same plaintext enciphering to the same ciphertext, by producing different key streams. - Selection of a small value for j will require more cycles through the encipherment algorithm per unit of plaintext and thus cause greater processing overheads. - Only multiples of j bits can be enciphered. - OFB mode of operation does not extend ciphertext errors in the resultant plaintext output. Every bit error in the ciphertext causes only one bit to be in error in the deciphered plaintext. - OFB mode is not self-synchronising. If the two operation of encipherment and decipherment get out of synchronism, the system needs to be re-initialised. - Each re-initialisation should use a value of the start variable different from the start variable values used before with the same key. The reason for this is that an identical bit stream would be produced each time from the same parameters. This would be susceptible to a ' known plaintext' attack. Triple ECB Mode (des_ecb3_encrypt()) - Encrypt with key1, decrypt with key2 and encrypt with key3 again. - As for ECB encryption but increases the key length to 168 bits. There are theoretic attacks that can be used that make the effective key length 112 bits, but this attack also requires 2^56 blocks of memory, not very likely, even for the NSA. - If both keys are the same it is equivalent to encrypting once with just one key. - If the first and last key are the same, the key length is 112 bits. There are attacks that could reduce the key space to 55 bit's but it requires 2^56 blocks of memory. - If all 3 keys are the same, this is effectively the same as normal ecb mode. Triple CBC Mode (des_ede3_cbc_encrypt()) - Encrypt with key1, decrypt with key2 and then encrypt with key3. - As for CBC encryption but increases the key length to 168 bits with the same restrictions as for triple ecb mode. ==== digest.doc ======================================================== The Message Digest subroutines. These routines require "evp.h" to be included. These functions are a higher level interface to the various message digest routines found in this library. As such, they allow the same code to be used to digest via different algorithms with only a change in an initial parameter. They are basically just a front-end to the MD2, MD5, SHA and SHA1 routines. These routines all take a pointer to the following structure to specify which message digest algorithm to use. typedef struct evp_md_st { int type; int pkey_type; int md_size; void (*init)(); void (*update)(); void (*final)(); int required_pkey_type; /*EVP_PKEY_xxx */ int (*sign)(); int (*verify)(); } EVP_MD; If additional message digest algorithms are to be supported, a structure of this type needs to be declared and populated and then the Digest routines can be used with that algorithm. The type field is the object NID of the digest type (read the section on Objects for an explanation). The pkey_type is the Object type to use when the a message digest is generated by there routines and then is to be signed with the pkey algorithm. Md_size is the size of the message digest returned. Init, update and final are the relevant functions to perform the message digest function by parts. One reason for specifying the message digest to use via this mechanism is that if you only use md5, only the md5 routines will be included in you linked program. If you passed an integer that specified which message digest to use, the routine that mapped that integer to a set of message digest functions would cause all the message digests functions to be link into the code. This setup also allows new message digest functions to be added by the application. The six message digests defined in this library are EVP_MD *EVP_md2(void); /* RSA sign/verify */ EVP_MD *EVP_md5(void); /* RSA sign/verify */ EVP_MD *EVP_sha(void); /* RSA sign/verify */ EVP_MD *EVP_sha1(void); /* RSA sign/verify */ EVP_MD *EVP_dss(void); /* DSA sign/verify */ EVP_MD *EVP_dss1(void); /* DSA sign/verify */ All the message digest routines take a EVP_MD_CTX pointer as an argument. The state of the message digest is kept in this structure. typedef struct pem_md_ctx_st { EVP_MD *digest; union { unsigned char base[4]; /* this is used in my library as a * 'pointer' to all union elements * structures. */ MD2_CTX md2; MD5_CTX md5; SHA_CTX sha; } md; } EVP_MD_CTX; The Digest functions are as follows. void EVP_DigestInit( EVP_MD_CTX *ctx, EVP_MD *type); This function is used to initialise the EVP_MD_CTX. The message digest that will associated with 'ctx' is specified by 'type'. void EVP_DigestUpdate( EVP_MD_CTX *ctx, unsigned char *data, unsigned int cnt); This function is used to pass more data to the message digest function. 'cnt' bytes are digested from 'data'. void EVP_DigestFinal( EVP_MD_CTX *ctx, unsigned char *md, unsigned int *len); This function finishes the digestion and puts the message digest into 'md'. The length of the message digest is put into len; EVP_MAX_MD_SIZE is the size of the largest message digest that can be returned from this function. Len can be NULL if the size of the digest is not required. ==== encode.doc ======================================================== void EVP_EncodeInit(EVP_ENCODE_CTX *ctx); void EVP_EncodeUpdate(EVP_ENCODE_CTX *ctx,unsigned char *out, int *outl,unsigned char *in,int inl); void EVP_EncodeFinal(EVP_ENCODE_CTX *ctx,unsigned char *out,int *outl); int EVP_EncodeBlock(unsigned char *t, unsigned char *f, int n); void EVP_DecodeInit(EVP_ENCODE_CTX *ctx); int EVP_DecodeUpdate(EVP_ENCODE_CTX *ctx,unsigned char *out,int *outl, unsigned char *in, int inl); int EVP_DecodeFinal(EVP_ENCODE_CTX *ctx, unsigned char *out, int *outl); int EVP_DecodeBlock(unsigned char *t, unsigned char *f, int n); ==== envelope.doc ======================================================== The following routines are use to create 'digital' envelopes. By this I mean that they perform various 'higher' level cryptographic functions. Have a read of 'cipher.doc' and 'digest.doc' since those routines are used by these functions. cipher.doc contains documentation about the cipher part of the envelope library and digest.doc contatins the description of the message digests supported. To 'sign' a document involves generating a message digest and then encrypting the digest with an private key. #define EVP_SignInit(a,b) EVP_DigestInit(a,b) #define EVP_SignUpdate(a,b,c) EVP_DigestUpdate(a,b,c) Due to the fact this operation is basically just an extended message digest, the first 2 functions are macro calls to Digest generating functions. int EVP_SignFinal( EVP_MD_CTX *ctx, unsigned char *md, unsigned int *s, EVP_PKEY *pkey); This finalisation function finishes the generation of the message digest and then encrypts the digest (with the correct message digest object identifier) with the EVP_PKEY private key. 'ctx' is the message digest context. 'md' will end up containing the encrypted message digest. This array needs to be EVP_PKEY_size(pkey) bytes long. 's' will actually contain the exact length. 'pkey' of course is the private key. It is one of EVP_PKEY_RSA or EVP_PKEY_DSA type. If there is an error, 0 is returned, otherwise 1. Verify is used to check an signed message digest. #define EVP_VerifyInit(a,b) EVP_DigestInit(a,b) #define EVP_VerifyUpdate(a,b,c) EVP_DigestUpdate(a,b,c) Since the first step is to generate a message digest, the first 2 functions are macros. int EVP_VerifyFinal( EVP_MD_CTX *ctx, unsigned char *md, unsigned int s, EVP_PKEY *pkey); This function finishes the generation of the message digest and then compares it with the supplied encrypted message digest. 'md' contains the 's' bytes of encrypted message digest. 'pkey' is used to public key decrypt the digest. It is then compared with the message digest just generated. If they match, 1 is returned else 0. int EVP_SealInit(EVP_CIPHER_CTX *ctx, EVP_CIPHER *type, unsigned char **ek, int *ekl, unsigned char *iv, EVP_PKEY **pubk, int npubk); Must have at least one public key, error is 0. I should also mention that the buffers pointed to by 'ek' need to be EVP_PKEY_size(pubk[n]) is size. #define EVP_SealUpdate(a,b,c,d,e) EVP_EncryptUpdate(a,b,c,d,e) void EVP_SealFinal(EVP_CIPHER_CTX *ctx,unsigned char *out,int *outl); int EVP_OpenInit(EVP_CIPHER_CTX *ctx,EVP_CIPHER *type,unsigned char *ek, int ekl,unsigned char *iv,EVP_PKEY *priv); 0 on failure #define EVP_OpenUpdate(a,b,c,d,e) EVP_DecryptUpdate(a,b,c,d,e) int EVP_OpenFinal(EVP_CIPHER_CTX *ctx, unsigned char *out, int *outl); Decrypt final return code ==== error.doc ======================================================== The error routines. The 'error' system I've implemented is intended to server 2 purpose, to record the reason why a command failed and to record where in the libraries the failure occurred. It is more or less setup to record a 'trace' of which library components were being traversed when the error occurred. When an error is recorded, it is done so a as single unsigned long which is composed of three parts. The top byte is the 'library' number, the middle 12 bytes is the function code, and the bottom 12 bits is the 'reason' code. Each 'library', or should a say, 'section' of the SSLeay library has a different unique 'library' error number. Each function in the library has a number that is unique for that library. Each 'library' also has a number for each 'error reason' that is only unique for that 'library'. Due to the way these error routines record a 'error trace', there is an array per thread that is used to store the error codes. The various functions in this library are used to access and manipulate this array. void ERR_put_error(int lib, int func,int reason); This routine records an error in library 'lib', function 'func' and reason 'reason'. As errors get 'put' into the buffer, they wrap around and overwrite old errors if too many are written. It is assumed that the last errors are the most important. unsigned long ERR_get_error(void ); This function returns the last error added to the error buffer. In effect it is popping the value off the buffer so repeated calls will continue to return values until there are no more errors to return in which case 0 is returned. unsigned long ERR_peek_error(void ); This function returns the value of the last error added to the error buffer but does not 'pop' it from the buffer. void ERR_clear_error(void ); This function clears the error buffer, discarding all unread errors. While the above described error system obviously produces lots of different error number, a method for 'reporting' these errors in a human readable form is required. To achieve this, each library has the option of 'registering' error strings. typedef struct ERR_string_data_st { unsigned long error; char *string; } ERR_STRING_DATA; The 'ERR_STRING_DATA' contains an error code and the corresponding text string. To add new function error strings for a library, the ERR_STRING_DATA needs to be 'registered' with the library. void ERR_load_strings(unsigned long lib,ERR_STRING_DATA *err); This function 'registers' the array of ERR_STRING_DATA pointed to by 'err' as error text strings for the error library 'lib'. void ERR_free_strings(void); This function free()s all the loaded error strings. char *ERR_error_string(unsigned long error,char *buf); This function returns a text string that is a human readable version of the error represented by 'error'. Buff should be at least 120 bytes long and if it is NULL, the return value is a pointer to a static variable that will contain the error string, otherwise 'buf' is returned. If there is not a text string registered for a particular error, a text string containing the error number is returned instead. void ERR_print_errors(BIO *bp); void ERR_print_errors_fp(FILE *fp); This function is a convenience routine that prints the error string for each error until all errors have been accounted for. char *ERR_lib_error_string(unsigned long e); char *ERR_func_error_string(unsigned long e); char *ERR_reason_error_string(unsigned long e); The above three functions return the 3 different components strings for the error 'e'. ERR_error_string() uses these functions. void ERR_load_ERR_strings(void ); This function 'registers' the error strings for the 'ERR' module. void ERR_load_crypto_strings(void ); This function 'register' the error strings for just about every library in the SSLeay package except for the SSL routines. There is no need to ever register any error text strings and you will probably save in program size. If on the other hand you do 'register' all errors, it is quite easy to determine why a particular routine failed. As a final footnote as to why the error system is designed as it is. 1) I did not want a single 'global' error code. 2) I wanted to know which subroutine a failure occurred in. 3) For Windows NT etc, it should be simple to replace the 'key' routines with code to pass error codes back to the application. 4) I wanted the option of meaningful error text strings. Late breaking news - the changes to support threads. Each 'thread' has an 'ERR_STATE' state associated with it. ERR_STATE *ERR_get_state(void ) will return the 'state' for the calling thread/process. ERR_remove_state(unsigned long pid); will 'free()' this state. If pid == 0 the current 'thread/process' will have it's error state removed. If you do not remove the error state of a thread, this could be considered a form of memory leak, so just after 'reaping' a thread that has died, call ERR_remove_state(pid). Have a read of thread.doc for more details for what is required for multi-threading support. All the other error routines will work correctly when using threads. ==== idea.doc ======================================================== The IDEA library. IDEA is a block cipher that operates on 64bit (8 byte) quantities. It uses a 128bit (16 byte) key. It can be used in all the modes that DES can be used. This library implements the ecb, cbc, cfb64 and ofb64 modes. For all calls that have an 'input' and 'output' variables, they can be the same. This library requires the inclusion of 'idea.h'. All of the encryption functions take what is called an IDEA_KEY_SCHEDULE as an argument. An IDEA_KEY_SCHEDULE is an expanded form of the idea key. For all modes of the IDEA algorithm, the IDEA_KEY_SCHEDULE used for decryption is different to the one used for encryption. The define IDEA_ENCRYPT is passed to specify encryption for the functions that require an encryption/decryption flag. IDEA_DECRYPT is passed to specify decryption. For some mode there is no encryption/decryption flag since this is determined by the IDEA_KEY_SCHEDULE. So to encrypt you would do the following idea_set_encrypt_key(key,encrypt_ks); idea_ecb_encrypt(...,encrypt_ks); idea_cbc_encrypt(....,encrypt_ks,...,IDEA_ENCRYPT); To Decrypt idea_set_encrypt_key(key,encrypt_ks); idea_set_decrypt_key(encrypt_ks,decrypt_ks); idea_ecb_encrypt(...,decrypt_ks); idea_cbc_encrypt(....,decrypt_ks,...,IDEA_DECRYPT); Please note that any of the encryption modes specified in my DES library could be used with IDEA. I have only implemented ecb, cbc, cfb64 and ofb64 for the following reasons. - ecb is the basic IDEA encryption. - cbc is the normal 'chaining' form for block ciphers. - cfb64 can be used to encrypt single characters, therefore input and output do not need to be a multiple of 8. - ofb64 is similar to cfb64 but is more like a stream cipher, not as secure (not cipher feedback) but it does not have an encrypt/decrypt mode. - If you want triple IDEA, thats 384 bits of key and you must be totally obsessed with security. Still, if you want it, it is simple enough to copy the function from the DES library and change the des_encrypt to idea_encrypt; an exercise left for the paranoid reader :-). The functions are as follows: void idea_set_encrypt_key( unsigned char *key; IDEA_KEY_SCHEDULE *ks); idea_set_encrypt_key converts a 16 byte IDEA key into an IDEA_KEY_SCHEDULE. The IDEA_KEY_SCHEDULE is an expanded form of the key which can be used to perform IDEA encryption. An IDEA_KEY_SCHEDULE is an expanded form of the key which is used to perform actual encryption. It can be regenerated from the IDEA key so it only needs to be kept when encryption is about to occur. Don't save or pass around IDEA_KEY_SCHEDULE's since they are CPU architecture dependent, IDEA keys are not. void idea_set_decrypt_key( IDEA_KEY_SCHEDULE *encrypt_ks, IDEA_KEY_SCHEDULE *decrypt_ks); This functions converts an encryption IDEA_KEY_SCHEDULE into a decryption IDEA_KEY_SCHEDULE. For all decryption, this conversion of the key must be done. In some modes of IDEA, an encryption/decryption flag is also required, this is because these functions involve block chaining and the way this is done changes depending on which of encryption of decryption is being done. Please note that there is no quick way to generate the decryption key schedule other than generating the encryption key schedule and then converting it. void idea_encrypt( unsigned long *data, IDEA_KEY_SCHEDULE *ks); This is the IDEA encryption function that gets called by just about every other IDEA routine in the library. You should not use this function except to implement 'modes' of IDEA. I say this because the functions that call this routine do the conversion from 'char *' to long, and this needs to be done to make sure 'non-aligned' memory access do not occur. Data is a pointer to 2 unsigned long's and ks is the IDEA_KEY_SCHEDULE to use. Encryption or decryption depends on the IDEA_KEY_SCHEDULE. void idea_ecb_encrypt( unsigned char *input, unsigned char *output, IDEA_KEY_SCHEDULE *ks); This is the basic Electronic Code Book form of IDEA (in DES this mode is called Electronic Code Book so I'm going to use the term for idea as well :-). Input is encrypted into output using the key represented by ks. Depending on the IDEA_KEY_SCHEDULE, encryption or decryption occurs. Input is 8 bytes long and output is 8 bytes. void idea_cbc_encrypt( unsigned char *input, unsigned char *output, long length, IDEA_KEY_SCHEDULE *ks, unsigned char *ivec, int enc); This routine implements IDEA in Cipher Block Chaining mode. Input, which should be a multiple of 8 bytes is encrypted (or decrypted) to output which will also be a multiple of 8 bytes. The number of bytes is in length (and from what I've said above, should be a multiple of 8). If length is not a multiple of 8, bad things will probably happen. ivec is the initialisation vector. This function updates iv after each call so that it can be passed to the next call to idea_cbc_encrypt(). void idea_cfb64_encrypt( unsigned char *in, unsigned char *out, long length, des_key_schedule ks, des_cblock *ivec, int *num, int enc); This is one of the more useful functions in this IDEA library, it implements CFB mode of IDEA with 64bit feedback. This allows you to encrypt an arbitrary number of bytes, you do not require 8 byte padding. Each call to this routine will encrypt the input bytes to output and then update ivec and num. Num contains 'how far' we are though ivec. Enc is used to indicate encryption or decryption. One very important thing to remember is that when decrypting, use the encryption form of the key. CFB64 mode operates by using the cipher to generate a stream of bytes which is used to encrypt the plain text. The cipher text is then encrypted to generate the next 64 bits to be xored (incrementally) with the next 64 bits of plain text. As can be seen from this, to encrypt or decrypt, the same 'cipher stream' needs to be generated but the way the next block of data is gathered for encryption is different for encryption and decryption. What this means is that to encrypt idea_set_encrypt_key(key,ks); idea_cfb64_encrypt(...,ks,..,IDEA_ENCRYPT) do decrypt idea_set_encrypt_key(key,ks) idea_cfb64_encrypt(...,ks,...,IDEA_DECRYPT) Note: The same IDEA_KEY_SCHEDULE but different encryption flags. For idea_cbc or idea_ecb, idea_set_decrypt_key() would need to be used to generate the IDEA_KEY_SCHEDULE for decryption. The reason I'm stressing this point is that I just wasted 3 hours today trying to decrypt using this mode and the decryption form of the key :-(. void idea_ofb64_encrypt( unsigned char *in, unsigned char *out, long length, des_key_schedule ks, des_cblock *ivec, int *num); This functions implements OFB mode of IDEA with 64bit feedback. This allows you to encrypt an arbitrary number of bytes, you do not require 8 byte padding. Each call to this routine will encrypt the input bytes to output and then update ivec and num. Num contains 'how far' we are though ivec. This is in effect a stream cipher, there is no encryption or decryption mode. The same key and iv should be used to encrypt and decrypt. For reading passwords, I suggest using des_read_pw_string() from my DES library. To generate a password from a text string, I suggest using MD5 (or MD2) to produce a 16 byte message digest that can then be passed directly to idea_set_encrypt_key(). ===== For more information about the specific IDEA modes in this library (ecb, cbc, cfb and ofb), read the section entitled 'Modes of DES' from the documentation on my DES library. What is said about DES is directly applicable for IDEA. ==== legal.doc ======================================================== From eay@mincom.com Thu Jun 27 00:25:45 1996 Received: by orb.mincom.oz.au id AA15821 (5.65c/IDA-1.4.4 for eay); Wed, 26 Jun 1996 14:25:45 +1000 Date: Wed, 26 Jun 1996 14:25:45 +1000 (EST) From: Eric Young <eay@mincom.oz.au> X-Sender: eay@orb To: Ken Toll <ktoll@ren.digitalage.com> Cc: Eric Young <eay@mincom.oz.au>, ssl-talk@netscape.com Subject: Re: Unidentified subject! In-Reply-To: <9606261950.ZM28943@ren.digitalage.com> Message-Id: <Pine.SOL.3.91.960626131156.28573K-100000@orb> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: O X-Status: This is a little off topic but since SSLeay is a free implementation of the SSLv2 protocol, I feel it is worth responding on the topic of if it is actually legal for Americans to use free cryptographic software. On Wed, 26 Jun 1996, Ken Toll wrote: > Is the U.S the only country that SSLeay cannot be used commercially > (because of RSAref) or is that going to be an issue with every country > that a client/server application (non-web browser/server) is deployed > and sold? >From what I understand, the software patents that apply to algorithms like RSA and DH only apply in the USA. The IDEA algorithm I believe is patened in europe (USA?), but considing how little it is used by other SSL implementations, it quite easily be left out of the SSLeay build (this can be done with a compile flag). Actually if the RSA patent did apply outside the USA, it could be rather interesting since RSA is not alowed to let RSA toolkits outside of the USA [1], and since these are the only forms that they will alow the algorithm to be used in, it would mean that non-one outside of the USA could produce public key software which would be a very strong statment for international patent law to make :-). This logic is a little flawed but it still points out some of the more interesting permutations of USA patent law and ITAR restrictions. Inside the USA there is also the unresolved issue of RC4/RC2 which were made public on sci.crypt in Sep 1994 (RC4) and Feb 1996 (RC2). I have copies of the origional postings if people are interested. RSA I believe claim that they were 'trade-secrets' and that some-one broke an NDA in revealing them. Other claim they reverse engineered the algorithms from compiled binaries. If the algorithms were reverse engineered, I believe RSA had no legal leg to stand on. If an NDA was broken, I don't know. Regardless, RSA, I believe, is willing to go to court over the issue so licencing is probably the best idea, or at least talk to them. If there are people who actually know more about this, pease let me know, I don't want to vilify or spread miss-information if I can help it. If you are not producing a web browser, it is easy to build SSLeay with RC2/RC4 removed. Since RC4 is the defacto standard cipher in all web software (and it is damn fast) it is more or less required for www use. For non www use of SSL, especially for an application where interoperability with other vendors is not critical just leave it out. Removing IDEA, RC2 and RC4 would only leave DES and Triple DES but they should be ok. Considing that Triple DES can encrypt at rates of 410k/sec on a pentium 100, and 940k/sec on a P6/200, this is quite reasonable performance. Single DES clocks in at 1160k/s and 2467k/s respectivly is actually quite fast for those not so paranoid (56 bit key).[1] > Is it possible to get a certificate for commercial use outside of the U.S.? yes. Thawte Consulting issues certificates (they are the people who sell the Sioux httpd server and are based in South Africa) Verisign will issue certificates for Sioux (sold from South Africa), so this proves that they will issue certificate for OS use if they are happy with the quality of the software. (The above mentioned companies just the ones that I know for sure are issuing certificates outside the USA). There is always the point that if you are using SSL for an intra net, SSLeay provides programs that can be used so you can issue your own certificates. They need polishing but at least it is a good starting point. I am not doing anything outside Australian law by implementing these algorithms (to the best of my knowedge). It is another example of how the world legal system does not cope with the internet very well. I may start making shared libraries available (I have now got DLL's for Windows). This will mean that distributions into the usa could be shipped with a version with a reduced cipher set and the versions outside could use the DLL/shared library with all the ciphers (and without RSAref). This could be completly hidden from the application, so this would not even require a re-linking. This is the reverse of what people were talking about doing to get around USA export regulations :-) eric [1]: The RSAref2.0 tookit is available on at least 3 ftp sites in Europe and one in South Africa. [2]: Since I always get questions when I post benchmark numbers :-), DES performace figures are in 1000's of bytes per second in cbc mode using an 8192 byte buffer. The pentium 100 was running Windows NT 3.51 DLLs and the 686/200 was running NextStep. I quote pentium 100 benchmarks because it is basically the 'entry level' computer that most people buy for personal use. Windows 95 is the OS shipping on those boxes, so I'll give NT numbers (the same Win32 runtime environment). The 686 numbers are present as an indication of where we will be in a few years. -- Eric Young | BOOL is tri-state according to Bill Gates. AARNet: eay@mincom.oz.au | RTFM Win32 GetMessage(). ==== lhash.doc ======================================================== The LHASH library. I wrote this library in 1991 and have since forgotten why I called it lhash. It implements a hash table from an article I read at the time from 'Communications of the ACM'. What makes this hash table different is that as the table fills, the hash table is increased (or decreased) in size via realloc(). When a 'resize' is done, instead of all hashes being redistributed over twice as many 'buckets', one bucket is split. So when an 'expand' is done, there is only a minimal cost to redistribute some values. Subsequent inserts will cause more single 'bucket' redistributions but there will never be a sudden large cost due to redistributing all the 'buckets'. The state for a particular hash table is kept in the LHASH structure. The LHASH structure also records statistics about most aspects of accessing the hash table. This is mostly a legacy of my writing this library for the reasons of implementing what looked like a nice algorithm rather than for a particular software product. Internal stuff you probably don't want to know about. The decision to increase or decrease the hash table size is made depending on the 'load' of the hash table. The load is the number of items in the hash table divided by the size of the hash table. The default values are as follows. If (hash->up_load < load) => expand. if (hash->down_load > load) => contract. The 'up_load' has a default value of 1 and 'down_load' has a default value of 2. These numbers can be modified by the application by just playing with the 'up_load' and 'down_load' variables. The 'load' is kept in a form which is multiplied by 256. So hash->up_load=8*256; will cause a load of 8 to be set. If you are interested in performance the field to watch is num_comp_calls. The hash library keeps track of the 'hash' value for each item so when a lookup is done, the 'hashes' are compared, if there is a match, then a full compare is done, and hash->num_comp_calls is incremented. If num_comp_calls is not equal to num_delete plus num_retrieve it means that your hash function is generating hashes that are the same for different values. It is probably worth changing your hash function if this is the case because even if your hash table has 10 items in a 'bucked', it can be searched with 10 'unsigned long' compares and 10 linked list traverses. This will be much less expensive that 10 calls to you compare function. LHASH *lh_new( unsigned long (*hash)(), int (*cmp)()); This function is used to create a new LHASH structure. It is passed function pointers that are used to store and retrieve values passed into the hash table. The 'hash' function is a hashing function that will return a hashed value of it's passed structure. 'cmp' is passed 2 parameters, it returns 0 is they are equal, otherwise, non zero. If there are any problems (usually malloc failures), NULL is returned, otherwise a new LHASH structure is returned. The hash value is normally truncated to a power of 2, so make sure that your hash function returns well mixed low order bits. void lh_free( LHASH *lh); This function free()s a LHASH structure. If there is malloced data in the hash table, it will not be freed. Consider using the lh_doall function to deallocate any remaining entries in the hash table. char *lh_insert( LHASH *lh, char *data); This function inserts the data pointed to by data into the lh hash table. If there is already and entry in the hash table entry, the value being replaced is returned. A NULL is returned if the new entry does not clash with an entry already in the table (the normal case) or on a malloc() failure (perhaps I should change this....). The 'char *data' is exactly what is passed to the hash and comparison functions specified in lh_new(). char *lh_delete( LHASH *lh, char *data); This routine deletes an entry from the hash table. The value being deleted is returned. NULL is returned if there is no such value in the hash table. char *lh_retrieve( LHASH *lh, char *data); If 'data' is in the hash table it is returned, else NULL is returned. The way these routines would normally be uses is that a dummy structure would have key fields populated and then ret=lh_retrieve(hash,&dummy);. Ret would now be a pointer to a fully populated structure. void lh_doall( LHASH *lh, void (*func)(char *a)); This function will, for every entry in the hash table, call function 'func' with the data item as parameters. This function can be quite useful when used as follows. void cleanup(STUFF *a) { STUFF_free(a); } lh_doall(hash,cleanup); lh_free(hash); This can be used to free all the entries, lh_free() then cleans up the 'buckets' that point to nothing. Be careful when doing this. If you delete entries from the hash table, in the call back function, the table may decrease in size, moving item that you are currently on down lower in the hash table. This could cause some entries to be skipped. The best solution to this problem is to set lh->down_load=0 before you start. This will stop the hash table ever being decreased in size. void lh_doall_arg( LHASH *lh; void(*func)(char *a,char *arg)); char *arg; This function is the same as lh_doall except that the function called will be passed 'arg' as the second argument. unsigned long lh_strhash( char *c); This function is a demo string hashing function. Since the LHASH routines would normally be passed structures, this routine would not normally be passed to lh_new(), rather it would be used in the function passed to lh_new(). The next three routines print out various statistics about the state of the passed hash table. These numbers are all kept in the lhash structure. void lh_stats( LHASH *lh, FILE *out); This function prints out statistics on the size of the hash table, how many entries are in it, and the number and result of calls to the routines in this library. void lh_node_stats( LHASH *lh, FILE *out); For each 'bucket' in the hash table, the number of entries is printed. void lh_node_usage_stats( LHASH *lh, FILE *out); This function prints out a short summary of the state of the hash table. It prints what I call the 'load' and the 'actual load'. The load is the average number of data items per 'bucket' in the hash table. The 'actual load' is the average number of items per 'bucket', but only for buckets which contain entries. So the 'actual load' is the average number of searches that will need to find an item in the hash table, while the 'load' is the average number that will be done to record a miss. ==== md2.doc ======================================================== The MD2 library. MD2 is a message digest algorithm that can be used to condense an arbitrary length message down to a 16 byte hash. The functions all need to be passed a MD2_CTX which is used to hold the MD2 context during multiple MD2_Update() function calls. The normal method of use for this library is as follows MD2_Init(...); MD2_Update(...); ... MD2_Update(...); MD2_Final(...); This library requires the inclusion of 'md2.h'. The main negative about MD2 is that it is slow, especially when compared to MD5. The functions are as follows: void MD2_Init( MD2_CTX *c); This function needs to be called to initiate a MD2_CTX structure for use. void MD2_Update( MD2_CTX *c; unsigned char *data; unsigned long len); This updates the message digest context being generated with 'len' bytes from the 'data' pointer. The number of bytes can be any length. void MD2_Final( unsigned char *md; MD2_CTX *c; This function is called when a message digest of the data digested with MD2_Update() is wanted. The message digest is put in the 'md' array and is MD2_DIGEST_LENGTH (16) bytes long. unsigned char *MD2( unsigned long n; unsigned char *d; unsigned char *md; This function performs a MD2_Init(), followed by a MD2_Update() followed by a MD2_Final() (using a local MD2_CTX). The resulting digest is put into 'md' if it is not NULL. Regardless of the value of 'md', the message digest is returned from the function. If 'md' was NULL, the message digest returned is being stored in a static structure. ==== md5.doc ======================================================== The MD5 library. MD5 is a message digest algorithm that can be used to condense an arbitrary length message down to a 16 byte hash. The functions all need to be passed a MD5_CTX which is used to hold the MD5 context during multiple MD5_Update() function calls. This library also contains random number routines that are based on MD5 The normal method of use for this library is as follows MD5_Init(...); MD5_Update(...); ... MD5_Update(...); MD5_Final(...); This library requires the inclusion of 'md5.h'. The functions are as follows: void MD5_Init( MD5_CTX *c); This function needs to be called to initiate a MD5_CTX structure for use. void MD5_Update( MD5_CTX *c; unsigned char *data; unsigned long len); This updates the message digest context being generated with 'len' bytes from the 'data' pointer. The number of bytes can be any length. void MD5_Final( unsigned char *md; MD5_CTX *c; This function is called when a message digest of the data digested with MD5_Update() is wanted. The message digest is put in the 'md' array and is MD5_DIGEST_LENGTH (16) bytes long. unsigned char *MD5( unsigned char *d; unsigned long n; unsigned char *md; This function performs a MD5_Init(), followed by a MD5_Update() followed by a MD5_Final() (using a local MD5_CTX). The resulting digest is put into 'md' if it is not NULL. Regardless of the value of 'md', the message digest is returned from the function. If 'md' was NULL, the message digest returned is being stored in a static structure. ==== memory.doc ======================================================== In the interests of debugging SSLeay, there is an option to compile using some simple memory leak checking. All malloc(), free() and realloc() calls in SSLeay now go via Malloc(), Free() and Realloc() (except those in crypto/lhash). If CRYPTO_MDEBUG is defined, these calls are #defined to CRYPTO_malloc(), CRYPTO_free() and CRYPTO_realloc(). If it is not defined, they are #defined to malloc(), free() and realloc(). the CRYPTO_malloc() routines by default just call the underlying library functons. If CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_ON) is called, memory leak detection is turned on. CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_OFF) turns it off. When turned on, each Malloc() or Realloc() call is recored along with the file and line number from where the call was made. (This is done using the lhash library which always uses normal system malloc(3) routines). void CRYPTO_mem_leaks(BIO *b); void CRYPTO_mem_leaks_fp(FILE *fp); These both print out the list of memory that has not been free()ed. This will probably be rather hard to read, but if you look for the 'top level' structure allocation, this will often give an idea as to what is not being free()ed. I don't expect people to use this stuff normally. ==== ca.1 ======================================================== From eay@orb.mincom.oz.au Thu Dec 28 23:56:45 1995 Received: by orb.mincom.oz.au id AA07374 (5.65c/IDA-1.4.4 for eay); Thu, 28 Dec 1995 13:56:45 +1000 Date: Thu, 28 Dec 1995 13:56:45 +1000 (EST) From: Eric Young <eay@mincom.oz.au> X-Sender: eay@orb To: sameer <sameer@c2.org> Cc: ssleay@mincom.oz.au Subject: Re: 'ca' In-Reply-To: <199512230440.UAA23410@infinity.c2.org> Message-Id: <Pine.SOL.3.91.951228133525.7269A-100000@orb> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO X-Status: On Fri, 22 Dec 1995, sameer wrote: > I could use documentation on 'ca'. Thanks. Very quickly. The ca program uses the ssleay.conf file for most of its configuration ./ca -help -verbose - Talk alot while doing things -config file - A config file. If you don't want to use the default config file -name arg - The particular CA definition to use In the config file, the section to use for parameters. This lets multiple setups to be contained in the one file. By default, the default_ca variable is looked up in the [ ca ] section. So in the shipped ssleay.conf, the CA definition used is CA_default. It could be any other name. -gencrl days - Generate a new CRL, days is when the next CRL is due This will generate a new certificate revocion list. -days arg - number of days to certify the certificate for When certifiying certificates, this is the number of days to use. -md arg - md to use, one of md2, md5, sha or sha1 -policy arg - The CA 'policy' to support I'll describe this later, but there are 2 policies definied in the shipped ssleay.conf -keyfile arg - PEM RSA private key file -key arg - key to decode the RSA private key if it is encrypted since we need to keep the CA's RSA key encrypted -cert - The CA certificate -in file - The input PEM encoded certificate request(s) -out file - Where to put the output file(s) -outdir dir - Where to put output certificates The -out options concatinates all the output certificied certificates to one file, -outdir puts them in a directory, named by serial number. -infiles .... - The last argument, requests to process The certificate requests to process, -in is the same. Just about all the above have default values defined in ssleay.conf. The key variables in ssleay.conf are (for the pariticular '-name' being used, in the default, it is CA_default). dir is where all the CA database stuff is kept. certs is where all the previously issued certificates are kept. The database is a simple text database containing the following tab separated fields. status: a value of 'R' - revoked, 'E' -expired or 'V' valid. issued date: When the certificate was certified. revoked date: When it was revoked, blank if not revoked. serial number: The certificate serial number. certificate: Where the certificate is located. CN: The name of the certificate. The demo file has quite a few made up values it it. The last 2 were added by the ca program and are acurate. The CA program does not update the 'certificate' file correctly right now. The serial field should be unique as should the CN/status combination. The ca program checks these at startup. What still needs to be wrtten is a program to 'regenerate' the data base file from the issued certificate list (and a CRL list). Back to the CA_default variables. Most of the variables are commented. policy is the default policy. Ok for policies, they define the order and which fields must be present in the certificate request and what gets filled in. So a value of countryName = match means that the country name must match the CA certificate. organizationalUnitName = optional The org.Unit,Name does not have to be present and commonName = supplied commonName must be supplied in the certificate request. For the 'policy_match' polocy, the order of the attributes in the generated certiticate would be countryName stateOrProvinceName organizationName organizationalUnitName commonName emailAddress Have a play, it sort of makes sense. If you think about how the persona requests operate, it is similar to the 'policy_match' policy and the 'policy_anything' is similar to what versign is doing. I hope this helps a bit. Some backend scripts are definitly needed to update the database and to make certificate revocion easy. All certificates issued should also be kept forever (or until they expire?) hope this helps eric (who has to run off an buy some cheap knee pads for the caving in 4 days time :-) -- Eric Young | Signature removed since it was generating AARNet: eay@mincom.oz.au | more followups than the message contents :-) ==== ms3-ca.doc ======================================================== Date: Mon, 9 Jun 97 08:00:33 +0200 From: Holger.Reif@PrakInf.TU-Ilmenau.DE (Holger Reif) Subject: ms3-ca.doc Organization: TU Ilmenau, Fak. IA, FG Telematik Content-Length: 14575 Status: RO X-Status: Loading client certs into MSIE 3.01 =================================== This document contains all the information necessary to successfully set up some scripts to issue client certs to Microsoft Internet Explorer. It includes the required knowledge about the model MSIE uses for client certification and includes complete sample scripts ready to play with. The scripts were tested against a modified ca program of SSLeay 0.6.6 and should work with the regular ca program that comes with version 0.8.0. I haven't tested against MSIE 4.0 You can use the information contained in this document in either way you want. However if you feel it saved you a lot of time I ask you to be as fair as to mention my name: Holger Reif <reif@prakinf.tu-ilmenau.de>. 1.) The model used by MSIE -------------------------- The Internet Explorer doesn't come with a embedded engine for installing client certs like Netscape's Navigator. It rather uses the CryptoAPI (CAPI) defined by Microsoft. CAPI comes with WindowsNT 4.0 or is installed together with Internet Explorer since 3.01. The advantage of this approach is a higher flexibility because the certificates in the (per user) system open certificate store may be used by other applications as well. The drawback however is that you need to do a bit more work to get a client cert issued. CAPI defines functions which will handle basic cryptographic work, eg. generating keys, encrypting some data, signing text or building a certificate request. The procedure is as follows: A CAPI function generates you a key pair and saves it into the certificate store. After that one builds a Distinguished Name. Together with that key pair another CAPI function forms a PKCS#10 request which you somehow need to submit to a CA. Finally the issued cert is given to a yet another CAPI function which saves it into the certificate store. The certificate store with the user's keys and certs is in the registry. You will find it under HKEY_CURRENT_USER/Software/Microsoft/Cryptography/ (I leave it to you as a little exercise to figure out what all the entries mean ;-). Note that the keys are protected only with the user's usual Windows login password. 2.) The practical usage ----------------------- Unfortunatly since CAPI is a system API you can't access its functions from HTML code directly. For this purpose Microsoft provides a wrapper called certenr3.dll. This DLL accesses the CAPI functions and provides an interface usable from Visual Basic Script. One needs to install that library on the computer which wants to have client cert. The easiest way is to load it as an ActiveX control (certenr3.dll is properly authenticode signed by MS ;-). If you have ever enrolled e cert request at a CA you will have installed it. At time of writing certenr3.dll is contained in http://www.microsoft.com/workshop/prog/security/csa/certenr3.exe. It comes with an README file which explains the available functions. It is labeled beta but every CA seems to use it anyway. The license.txt allows you the usage for your own purposes (as far as I understood) and a somehow limited distribution. The two functions of main interest are GenerateKeyPair and AcceptCredentials. For complete explanation of all possible parameters see the README file. Here are only minimal required parameters and their values. GenerateKeyPair(sessionID, FASLE, szName, 0, "ClientAuth", TRUE, FALSE, 1) - sessionID is a (locally to that computer) unique string to correlate the generated key pair with a cert installed later. - szName is the DN of the form "C=DE; S=Thueringen; L=Ilmenau; CN=Holger Reif; 1.2.840.113549.1.9.1=reif@prakinf.tu-ilmenau.de". Note that S is the abreviation for StateOrProvince. The recognized abreviation include CN, O, C, OU, G, I, L, S, T. If the abreviation is unknown (eg. for PKCS#9 email addr) you need to use the full object identifier. The starting point for searching them could be crypto/objects.h since all OIDs know to SSLeay are listed there. - note: the possible ninth parameter which should give a default name to the certificate storage location doesn't seem to work. Changes to the constant values in the call above doesn't seem to make sense. You can't generate PKCS#10 extensions with that function. The result of GenerateKeyPair is the base64 encoded PKCS#10 request. However it has a little strange format that SSLeay doesn't accept. (BTW I feel the decision of rejecting that format as standard conforming.) It looks like follows: 1st line with 76 chars 2nd line with 76 chars ... (n-2)th line with 76 chars (n-1)th line contains a multiple of 4 chars less then 76 (possible empty) (n)th line has zero or 4 chars (then with 1 or 2 equal signs - the original text's lenght wasn'T a multiple of 3) The line separator has two chars: 0x0d 0x0a AcceptCredentials(sessionID, credentials, 0, FALSE) - sessionID needs to be the same as while generating the key pair - credentials is the base64 encoded PKCS#7 object containing the cert. CRL's and CA certs are not required simply just the client cert. (It seems to me that both are not even checked somehow.) The only format of the base64 encoded object I succesfully used was all characters in a very long string without line feeds or carriage returns. (Hey, it doesn't matter, only a computer reads it!) The result should be S_OK. For error handling see the example that comes with certenr3.dll. A note about ASN.1 character encodings. certenr3.dll seems to know only about 2 of them: UniversalString and PrintableString. First it is definitely wrong for an email address which is IA5STRING (checked by ssleay's ca). Second unfortunately MSIE (at least until version 3.02) can't handle UniversalString correctly - they just blow up you cert store! Therefore ssleay's ca (starting from version 0.8.0) tries to convert the encodings automatically to IA5STRING or TeletexString. The beef is it will work only for the latin-1 (western) charset. Microsoft still has to do abit of homework... 3.) An example -------------- At least you need two steps: generating the key & request and then installing the certificate. A real world CA would have some more steps involved, eg. accepting some license. Note that both scripts shown below are just experimental state without any warrenty! First how to generate a request. Note that we can't use a static page because of the sessionID. I generate it from system time plus pid and hope it is unique enough. Your are free to feed it through md5 to get more impressive ID's ;-) Then the intended text is read in with sed which inserts the sessionID. -----BEGIN ms-enroll.cgi----- #!/bin/sh SESSION_ID=`date '+%y%m%d%H%M%S'`$$ echo Content-type: text/html echo sed s/template_for_sessId/$SESSION_ID/ <<EOF <HTML><HEAD> <TITLE>Certificate Enrollment Test Page</TITLE> </HEAD><BODY> <OBJECT classid="clsid:33BEC9E0-F78F-11cf-B782-00C04FD7BF43" codebase=certenr3.dll id=certHelper > </OBJECT> <CENTER> <H2>enrollment for a personal cert</H2> <BR><HR WIDTH=50%><BR><P> <FORM NAME="MSIE_Enrollment" ACTION="ms-gencert.cgi" ENCTYPE=x-www-form- encoded METHOD=POST> <TABLE> <TR><TD>Country</TD><TD><INPUT NAME="Country" VALUE=""></TD></TR> <TR><TD>State</TD><TD><INPUT NAME="StateOrProvince" VALUE=""></TD></TR> <TR><TD>Location</TD><TD><INPUT NAME="Location" VALUE=""></TD></TR> <TR><TD>Organization</TD><TD><INPUT NAME="Organization" VALUE=""></TD></TR> <TR><TD>Organizational Unit</TD> <TD><INPUT NAME="OrganizationalUnit" VALUE=""></TD></TR> <TR><TD>Name</TD><TD><INPUT NAME="CommonName" VALUE=""></TD></TR> <TR><TD>eMail Address</TD> <TD><INPUT NAME="EmailAddress" VALUE=""></TD></TR> <TR><TD></TD> <TD><INPUT TYPE="BUTTON" NAME="submit" VALUE="Beantragen"></TD></TR> </TABLE> <INPUT TYPE="hidden" NAME="SessionId" VALUE="template_for_sessId"> <INPUT TYPE="hidden" NAME="Request" VALUE=""> </FORM> <BR><HR WIDTH=50%><BR><P> </CENTER> <SCRIPT LANGUAGE=VBS> Dim DN Sub Submit_OnClick Dim TheForm Set TheForm = Document.MSIE_Enrollment sessionId = TheForm.SessionId.value reqHardware = FALSE C = TheForm.Country.value SP = TheForm.StateOrProvince.value L = TheForm.Location.value O = TheForm.Organization.value OU = TheForm.OrganizationalUnit.value CN = TheForm.CommonName.value Email = TheForm.EmailAddress.value szPurpose = "ClientAuth" doAcceptanceUINow = FALSE doOnline = TRUE DN = "" Call Add_RDN("C", C) Call Add_RDN("S", SP) Call Add_RDN("L", L) Call Add_RDN("O", O) Call Add_RDN("OU", OU) Call Add_RDN("CN", CN) Call Add_RDN("1.2.840.113549.1.9.1", Email) ' rsadsi ' pkcs ' pkcs9 ' eMailAddress On Error Resume Next sz10 = certHelper.GenerateKeyPair(sessionId, _ FALSE, DN, 0, ClientAuth, FASLE, TRUE, 1)_ theError = Err.Number On Error Goto 0 if (sz10 = Empty OR theError <> 0) Then sz = "The error '" & Hex(theError) & "' occurred." & chr(13) & _ chr(10) & "Your credentials could not be generated." result = MsgBox(sz, 0, "Credentials Enrollment") Exit Sub else TheForm.Request.value = sz10 TheForm.Submit end if End Sub Sub Add_RDN(sn, value) if (value <> "") then if (DN <> "") then DN = DN & "; " end if DN = DN & sn & "=" & value end if End Sub </SCRIPT> </BODY> </HTML> EOF -----END ms-enroll.cgi----- Second, how to extract the request and feed the certificate back? We need to "normalize" the base64 encoding of the PKCS#10 format which means regenerating the lines and wrapping with BEGIN and END line. This is done by gawk. The request is taken by ca the normal way. Then the cert needs to be packed into a PKCS#7 structure (note: the use of a CRL is necessary for crl2pkcs7 as of version 0.6.6. Starting with 0.8.0 it it might probably be ommited). Finally we need to format the PKCS#7 object and generate the HTML text. I use two templates to have a clearer script. 1st note: postit2 is slightly modified from a program I found at ncsa's ftp site. Grab it from http://www.easterngraphics.com/certs/IX9704/postit2.c. You need utils.c from there too. 2nd note: I'm note quite sure wether the gawk script really handles all possible inputs for the request right! Today I don't use this construction anymore myself. 3d note: the cert must be of version 3! This could be done with the nsComment line in ssleay.cnf... ------BEGIN ms-gencert.cgi----- #!/bin/sh FILE="/tmp/"`date '+%y%m%d%H%M%S'-`$$ rm -f "$FILE".* HOME=`pwd`; export HOME # as ssleay.cnf insists on having such an env var cd /usr/local/ssl #where demoCA (as named in ssleay.conf) is located postit2 -s " " -i 0x0d > "$FILE".inp # process the FORM vars SESSION_ID=`gawk '$1 == "SessionId" { print $2; exit }' "$FILE".inp` gawk \ 'BEGIN { \ OFS = ""; \ print "-----BEGIN CERTIFICATE REQUEST-----"; \ req_seen=0 \ } \ $1 == "Request" { \ req_seen=1; \ if (length($2) == 72) print($2); \ lastline=$2; \ next; \ } \ { \ if (req_seen == 1) { \ if (length($1) >= 72) print($1); \ else if (length(lastline) < 72) { \ req_seen=0; \ print (lastline,$1); \ } \ lastline=$1; \ } \ } \ END { \ print "-----END CERTIFICATE REQUEST-----"; \ }' > "$FILE".pem < "$FILE".inp ssleay ca -batch -in "$FILE".pem -key passwd -out "$FILE".out ssleay crl2pkcs7 -certfile "$FILE".out -out "$FILE".pkcs7 -in demoCA/crl.pem sed s/template_for_sessId/$SESSION_ID/ <ms-enroll2a.html >"$FILE".cert /usr/local/bin/gawk \ 'BEGIN { \ OFS = ""; \ dq = sprintf("%c",34); \ } \ $0 ~ "PKCS7" { next; } \ { \ print dq$0dq" & _"; \ }' <"$FILE".pkcs7 >> "$FILE".cert cat ms-enroll2b.html >>"$FILE".cert echo Content-type: text/html echo Content-length: `wc -c "$FILE".cert` echo cat "$FILE".cert rm -f "$FILE".* -----END ms-gencert.cgi----- ----BEGIN ms-enroll2a.html---- <HTML><HEAD><TITLE>Certificate Acceptance Test Page</TITLE></HEAD><BODY> <OBJECT classid="clsid:33BEC9E0-F78F-11cf-B782-00C04FD7BF43" codebase=certenr3.dll id=certHelper > </OBJECT> <CENTER> <H2>Your personal certificate</H2> <BR><HR WIDTH=50%><BR><P> Press the button! <P><INPUT TYPE=BUTTON VALUE="Nimm mich!" NAME="InstallCert"> </CENTER> <BR><HR WIDTH=50%><BR> <SCRIPT LANGUAGE=VBS> Sub InstallCert_OnClick sessionId = "template_for_sessId" credentials = "" & _ ----END ms-enroll2a.html---- ----BEGIN ms-enroll2b.html---- "" On Error Resume Next result = certHelper.AcceptCredentials(sessionId, credentials, 0, FALSE) if (IsEmpty(result)) Then sz = "The error '" & Err.Number & "' occurred." & chr(13) & chr(10) & "This Digital ID could not be registered." msgOut = MsgBox(sz, 0, "Credentials Registration Error") navigate "error.html" else sz = "Digital ID successfully registered." msgOut = MsgBox(sz, 0, "Credentials Registration") navigate "success.html" end if Exit Sub End Sub </SCRIPT> </BODY> </HTML> ----END ms-enroll2b.html---- 4.) What do do with the cert? ----------------------------- The cert is visible (without restarting MSIE) under the following menu: View->Options->Security->Personal certs. You can examine it's contents at least partially. To use it for client authentication you need to use SSL3.0 (fortunately SSLeay supports it with 0.8.0). Furthermore MSIE is told to only supports a kind of automatic selection of certs (I personally wasn't able to test it myself). But there is a requirement that the issuer of the server cert and the issuer of the client cert needs to be the same (according to a developer from MS). Which means: you need may more then one cert to talk to all servers... I'm sure we will get a bit more experience after ApacheSSL is available for SSLeay 0.8.8. I hope you enjoyed reading and that in future questions on this topic will rarely appear on ssl-users@moncom.com ;-) Ilmenau, 9th of June 1997 Holger Reif <reif@prakinf.tu-ilmenau.de> -- read you later - Holger Reif ---------------------------------------- Signaturprojekt Deutsche Einheit TU Ilmenau - Informatik - Telematik (Verdamp lang her) Holger.Reif@PrakInf.TU-Ilmenau.DE Alt wie ein Baum werden, um ueber http://Remus.PrakInf.TU-Ilmenau.DE/Reif/ alle 7 Bruecken gehen zu koennen ==== ns-ca.doc ======================================================== The following documentation was supplied by Jeff Barber, who provided the patch to the CA program to add this functionality. eric -- Jeff Barber Email: jeffb@issl.atl.hp.com Hewlett Packard Phone: (404) 648-9503 Internet and System Security Lab Fax: (404) 648-9516 oo ---------------------cut /\ here for ns-ca.doc ------------------------------ This document briefly describes how to use SSLeay to implement a certificate authority capable of dynamically serving up client certificates for version 3.0 beta 5 (and presumably later) versions of the Netscape Navigator. Before describing how this is done, it's important to understand a little about how the browser implements its client certificate support. This is documented in some detail in the URLs based at <URL:http://home.netscape.com/eng/security/certs.html>. Here's a brief overview: - The Navigator supports a new HTML tag "KEYGEN" which will cause the browser to generate an RSA key pair when you submit a form containing the tag. The public key, along with an optional challenge (supposedly provided for use in certificate revocation but I don't use it) is signed, DER-encoded, base-64 encoded and sent to the web server as the value of the variable whose NAME is provided in the KEYGEN tag. The private key is stored by the browser in a local key database. This "Signed Public Key And Challenge" (SPKAC) arrives formatted into 64 character lines (which are of course URL-encoded when sent via HTTP -- i.e. spaces, newlines and most punctuatation are encoded as "%HH" where HH is the hex equivalent of the ASCII code). Note that the SPKAC does not contain the other usual attributes of a certificate request, especially the subject name fields. These must be otherwise encoded in the form for submission along with the SPKAC. - Either immediately (in response to this form submission), or at some later date (a real CA will probably verify your identity in some way before issuing the certificate), a web server can send a certificate based on the public key and other attributes back to the browser by encoding it in DER (the binary form) and sending it to the browser as MIME type: "Content-type: application/x-x509-user-cert" The browser uses the public key encoded in the certificate to associate the certificate with the appropriate private key in its local key database. Now, the certificate is "installed". - When a server wants to require authentication based on client certificates, it uses the right signals via the SSL protocol to trigger the Navigator to ask you which certificate you want to send. Whether the certificate is accepted is dependent on CA certificates and so forth installed in the server and is beyond the scope of this document. Now, here's how the SSLeay package can be used to provide client certficates: - You prepare a file for input to the SSLeay ca application. The file contains a number of "name = value" pairs that identify the subject. The names here are the same subject name component identifiers used in the CA section of the lib/ssleay.conf file, such as "emailAddress", "commonName" "organizationName" and so forth. Both the long version and the short version (e.g. "Email", "CN", "O") can be used. One more name is supported: this one is "SPKAC". Its value is simply the value of the base-64 encoded SPKAC sent by the browser (with all the newlines and other space charaters removed -- and newline escapes are NOT supported). [ As of SSLeay 0.6.4, multiple lines are supported. Put a \ at the end of each line and it will be joined with the previous line with the '\n' removed - eay ] Here's a sample input file: C = US SP = Georgia O = Some Organization, Inc. OU = Netscape Compatibility Group CN = John X. Doe Email = jxdoe@someorg.com SPKAC = MIG0MGAwXDANBgkqhkiG9w0BAQEFAANLADBIAkEAwmk6FMJ4uAVIYbcvIOx5+bDGTfvL8X5gE+R67ccMk6rCSGbVQz2cetyQtnI+VIs0NwdD6wjuSuVtVFbLoHonowIDAQABFgAwDQYJKoZIhvcNAQEEBQADQQBFZDUWFl6BJdomtN1Bi53mwijy1rRgJ4YirF15yBEDM3DjAQkKXHYOIX+qpz4KXKnl6EYxTnGSFL5wWt8X2iyx - You execute the ca command (either from a CGI program run out of the web server, or as a later manual task) giving it the above file as input. For example, if the file were named /tmp/cert.req, you'd run: $SSLDIR/bin/ca -spkac /tmp/cert.req -out /tmp/cert The output is in DER format (binary) if a -out argument is provided, as above; otherwise, it's in the PEM format (base-64 encoded DER). Also, the "-batch" switch is implied by the "-spkac" so you don't get asked whether to complete the signing (probably it shouldn't work this way but I was only interested in hacking together an online CA that could be used for issuing test certificates). The "-spkac" capability doesn't support multiple files (I think). Any CHALLENGE provided in the SPKAC is simply ignored. The interactions between the identification fields you provide and those identified in your lib/ssleay.conf are the same as if you did an ordinary "ca -in infile -out outfile" -- that is, if something is marked as required in the ssleay.conf file and it isn't found in the -spkac file, the certificate won't be issued. - Now, you pick up the output from /tmp/cert and pass it back to the Navigator prepending the Content-type string described earlier. - In order to run the ca command out of a CGI program, you must provide a password to decrypt the CA's private key. You can do this by using "echo MyKeyPassword | $SSLDIR/bin/ca ..." I think there's a way to not encrypt the key file in the first place, but I didn't see how to do that, so I made a small change to the library that allows the password to be accepted from a pipe. Either way is UTTERLY INSECURE and a real CA would never do that. [ You can use the 'ssleay rsa' command to remove the password from the private key, or you can use the '-key' option to the ca command to specify the decryption key on the command line or use the -nodes option when generating the key. ca will try to clear the command line version of the password but for quite a few operating systems, this is not possible. - eric ] So, what do you have to do to make use of this stuff to create an online demo CA capability with SSLeay? 1 Create an HTML form for your users. The form should contain fields for all of the required or optional fields in ssleay.conf. The form must contain a KEYGEN tag somewhere with at least a NAME attribute. 2 Create a CGI program to process the form input submitted by the browser. The CGI program must URL-decode the variables and create the file described above, containing subject identification info as well as the SPKAC block. It should then run the the ca program with the -spkac option. If it works (check the exit status), return the new certificate with the appropriate MIME type. If not, return the output of the ca command with MIME type "text/plain". 3 Set up your web server to accept connections signed by your demo CA. This probably involves obtaining the PEM-encoded CA certificate (ordinarily in $SSLDIR/CA/cacert.pem) and installing it into a server database. See your server manual for instructions. ==== obj.doc ======================================================== The Object library. As part of my Crypto library, I found I required a method of identifying various objects. These objects normally had 3 different values associated with them, a short text name, a long (or lower case) text name, and an ASN.1 Object Identifier (which is a sequence of numbers). This library contains a static list of objects and functions to lookup according to one type and to return the other types. To use these routines, 'Object.h' needs to be included. For each supported object, #define entries are defined as follows #define SN_Algorithm "Algorithm" #define LN_algorithm "algorithm" #define NID_algorithm 38 #define OBJ_algorithm 1L,3L,14L,3L,2L SN_ stands for short name. LN_ stands for either long name or lowercase name. NID_ stands for Numeric ID. I each object has a unique NID and this should be used internally to identify objects. OBJ_ stands for ASN.1 Object Identifier or ASN1_OBJECT as defined in the ASN1 routines. These values are used in ASN1 encoding. The following functions are to be used to return pointers into a static definition of these types. What this means is "don't try to free() any pointers returned from these functions. ASN1_OBJECT *OBJ_nid2obj( int n); Return the ASN1_OBJECT that corresponds to a NID of n. char *OBJ_nid2ln( int n); Return the long/lower case name of the object represented by the NID of n. char *OBJ_nid2sn( int n); Return the short name for the object represented by the NID of n. ASN1_OBJECT *OBJ_dup( ASN1_OBJECT *o); Duplicate and return a new ASN1_OBJECT that is the same as the passed parameter. int OBJ_obj2nid( ASN1_OBJECT *o); Given ASN1_OBJECT o, return the NID that corresponds. int OBJ_ln2nid( char *s); Given the long/lower case name 's', return the NID of the object. int OBJ_sn2nid( char *s); Given the short name 's', return the NID of the object. char *OBJ_bsearch( char *key, char *base, int num, int size, int (*cmp)()); Since I have come across a few platforms that do not have the bsearch() function, OBJ_bsearch is my version of that function. Feel free to use this function, but you may as well just use the normal system bsearch(3) if it is present. This version also has tolerance of being passed NULL pointers. ==== keys =========================================================== EVP_PKEY_DSA EVP_PKEY_DSA2 EVP_PKEY_DSA3 EVP_PKEY_DSA4 EVP_PKEY_RSA EVP_PKEY_RSA2 valid DSA pkey types NID_dsa NID_dsaWithSHA NID_dsaWithSHA1 NID_dsaWithSHA1_2 valid RSA pkey types NID_rsaEncryption NID_rsa NID_dsaWithSHA NID_dsaWithSHA DSA SHA NID_dsa NID_dsaWithSHA1 DSA SHA1 NID_md2 NID_md2WithRSAEncryption RSA-pkcs1 MD2 NID_md5 NID_md5WithRSAEncryption RSA-pkcs1 MD5 NID_mdc2 NID_mdc2WithRSA RSA-none MDC2 NID_ripemd160 NID_ripemd160WithRSA RSA-pkcs1 RIPEMD160 NID_sha NID_shaWithRSAEncryption RSA-pkcs1 SHA NID_sha1 NID_sha1WithRSAEncryption RSA-pkcs1 SHA1 ==== rand.doc ======================================================== My Random number library. These routines can be used to generate pseudo random numbers and can be used to 'seed' the pseudo random number generator (RNG). The RNG make no effort to reproduce the same random number stream with each execution. Various other routines in the SSLeay library 'seed' the RNG when suitable 'random' input data is available. Read the section at the end for details on the design of the RNG. void RAND_bytes( unsigned char *buf, int num); This routine puts 'num' random bytes into 'buf'. One should make sure RAND_seed() has been called before using this routine. void RAND_seed( unsigned char *buf, int num); This routine adds more 'seed' data the RNG state. 'num' bytes are added to the RNG state, they are taken from 'buf'. This routine can be called with sensitive data such as user entered passwords. This sensitive data is in no way recoverable from the RAND library routines or state. Try to pass as much data from 'random' sources as possible into the RNG via this function. Also strongly consider using the RAND_load_file() and RAND_write_file() routines. void RAND_cleanup(); When a program has finished with the RAND library, if it so desires, it can 'zero' all RNG state. The following 3 routines are convenience routines that can be used to 'save' and 'restore' data from/to the RNG and it's state. Since the more 'random' data that is feed as seed data the better, why not keep it around between executions of the program? Of course the application should pass more 'random' data in via RAND_seed() and make sure no-one can read the 'random' data file. char *RAND_file_name( char *buf, int size); This routine returns a 'default' name for the location of a 'rand' file. The 'rand' file should keep a sequence of random bytes used to initialise the RNG. The filename is put in 'buf'. Buf is 'size' bytes long. Buf is returned if things go well, if they do not, NULL is returned. The 'rand' file name is generated in the following way. First, if there is a 'RANDFILE' environment variable, it is returned. Second, if there is a 'HOME' environment variable, $HOME/.rand is returned. Third, NULL is returned. NULL is also returned if a buf would overflow. int RAND_load_file( char *file, long number); This function 'adds' the 'file' into the RNG state. It does this by doing a RAND_seed() on the value returned from a stat() system call on the file and if 'number' is non-zero, upto 'number' bytes read from the file. The number of bytes passed to RAND_seed() is returned. int RAND_write_file( char *file), RAND_write_file() writes N random bytes to the file 'file', where N is the size of the internal RND state (currently 1k). This is a suitable method of saving RNG state for reloading via RAND_load_file(). What follows is a description of this RNG and a description of the rational behind it's design. It should be noted that this RNG is intended to be used to generate 'random' keys for various ciphers including generation of DH and RSA keys. It should also be noted that I have just created a system that I am happy with. It may be overkill but that does not worry me. I have not spent that much time on this algorithm so if there are glaring errors, please let me know. Speed has not been a consideration in the design of these routines. First up I will state the things I believe I need for a good RNG. 1) A good hashing algorithm to mix things up and to convert the RNG 'state' to random numbers. 2) An initial source of random 'state'. 3) The state should be very large. If the RNG is being used to generate 4096 bit RSA keys, 2 2048 bit random strings are required (at a minimum). If your RNG state only has 128 bits, you are obviously limiting the search space to 128 bits, not 2048. I'm probably getting a little carried away on this last point but it does indicate that it may not be a bad idea to keep quite a lot of RNG state. It should be easier to break a cipher than guess the RNG seed data. 4) Any RNG seed data should influence all subsequent random numbers generated. This implies that any random seed data entered will have an influence on all subsequent random numbers generated. 5) When using data to seed the RNG state, the data used should not be extractable from the RNG state. I believe this should be a requirement because one possible source of 'secret' semi random data would be a private key or a password. This data must not be disclosed by either subsequent random numbers or a 'core' dump left by a program crash. 6) Given the same initial 'state', 2 systems should deviate in their RNG state (and hence the random numbers generated) over time if at all possible. 7) Given the random number output stream, it should not be possible to determine the RNG state or the next random number. The algorithm is as follows. There is global state made up of a 1023 byte buffer (the 'state'), a working message digest ('md') and a counter ('count'). Whenever seed data is added, it is inserted into the 'state' as follows. The input is chopped up into units of 16 bytes (or less for the last block). Each of these blocks is run through the MD5 message digest. The data passed to the MD5 digest is the current 'md', the same number of bytes from the 'state' (the location determined by in incremented looping index) as the current 'block' and the new key data 'block'. The result of this is kept in 'md' and also xored into the 'state' at the same locations that were used as input into the MD5. I believe this system addresses points 1 (MD5), 3 (the 'state'), 4 (via the 'md'), 5 (by the use of MD5 and xor). When bytes are extracted from the RNG, the following process is used. For each group of 8 bytes (or less), we do the following, Input into MD5, the top 8 bytes from 'md', the byte that are to be overwritten by the random bytes and bytes from the 'state' (incrementing looping index). From this digest output (which is kept in 'md'), the top (upto) 8 bytes are returned to the caller and the bottom (upto) 8 bytes are xored into the 'state'. Finally, after we have finished 'generation' random bytes for the called, 'count' (which is incremented) and 'md' are fed into MD5 and the results are kept in 'md'. I believe the above addressed points 1 (use of MD5), 6 (by hashing into the 'state' the 'old' data from the caller that is about to be overwritten) and 7 (by not using the 8 bytes given to the caller to update the 'state', but they are used to update 'md'). So of the points raised, only 2 is not addressed, but sources of random data will always be a problem. ==== rc2.doc ======================================================== The RC2 library. RC2 is a block cipher that operates on 64bit (8 byte) quantities. It uses variable size key, but 128bit (16 byte) key would normally be considered good. It can be used in all the modes that DES can be used. This library implements the ecb, cbc, cfb64, ofb64 modes. I have implemented this library from an article posted to sci.crypt on 11-Feb-1996. I personally don't know how far to trust the RC2 cipher. While it is capable of having a key of any size, not much reseach has publically been done on it at this point in time (Apr-1996) since the cipher has only been public for a few months :-) It is of a similar speed to DES and IDEA, so unless it is required for meeting some standard (SSLv2, perhaps S/MIME), it would probably be advisable to stick to IDEA, or for the paranoid, Tripple DES. Mind you, having said all that, I should mention that I just read alot and implement ciphers, I'm a 'babe in the woods' when it comes to evaluating ciphers :-). For all calls that have an 'input' and 'output' variables, they can be the same. This library requires the inclusion of 'rc2.h'. All of the encryption functions take what is called an RC2_KEY as an argument. An RC2_KEY is an expanded form of the RC2 key. For all modes of the RC2 algorithm, the RC2_KEY used for decryption is the same one that was used for encryption. The define RC2_ENCRYPT is passed to specify encryption for the functions that require an encryption/decryption flag. RC2_DECRYPT is passed to specify decryption. Please note that any of the encryption modes specified in my DES library could be used with RC2. I have only implemented ecb, cbc, cfb64 and ofb64 for the following reasons. - ecb is the basic RC2 encryption. - cbc is the normal 'chaining' form for block ciphers. - cfb64 can be used to encrypt single characters, therefore input and output do not need to be a multiple of 8. - ofb64 is similar to cfb64 but is more like a stream cipher, not as secure (not cipher feedback) but it does not have an encrypt/decrypt mode. - If you want triple RC2, thats 384 bits of key and you must be totally obsessed with security. Still, if you want it, it is simple enough to copy the function from the DES library and change the des_encrypt to RC2_encrypt; an exercise left for the paranoid reader :-). The functions are as follows: void RC2_set_key( RC2_KEY *ks; int len; unsigned char *key; int bits; RC2_set_key converts an 'len' byte key into a RC2_KEY. A 'ks' is an expanded form of the 'key' which is used to perform actual encryption. It can be regenerated from the RC2 key so it only needs to be kept when encryption or decryption is about to occur. Don't save or pass around RC2_KEY's since they are CPU architecture dependent, 'key's are not. RC2 is an interesting cipher in that it can be used with a variable length key. 'len' is the length of 'key' to be used as the key. A 'len' of 16 is recomended. The 'bits' argument is an interesting addition which I only found out about in Aug 96. BSAFE uses this parameter to 'limit' the number of bits used for the key. To use the 'key' unmodified, set bits to 1024. This is what old versions of my RC2 library did (SSLeay 0.6.3). RSAs BSAFE library sets this parameter to be 128 if 128 bit keys are being used. So to be compatable with BSAFE, set it to 128, if you don't want to reduce RC2's key length, leave it at 1024. void RC2_encrypt( unsigned long *data, RC2_KEY *key, int encrypt); This is the RC2 encryption function that gets called by just about every other RC2 routine in the library. You should not use this function except to implement 'modes' of RC2. I say this because the functions that call this routine do the conversion from 'char *' to long, and this needs to be done to make sure 'non-aligned' memory access do not occur. Data is a pointer to 2 unsigned long's and key is the RC2_KEY to use. Encryption or decryption is indicated by 'encrypt'. which can have the values RC2_ENCRYPT or RC2_DECRYPT. void RC2_ecb_encrypt( unsigned char *in, unsigned char *out, RC2_KEY *key, int encrypt); This is the basic Electronic Code Book form of RC2 (in DES this mode is called Electronic Code Book so I'm going to use the term for rc2 as well. Input is encrypted into output using the key represented by key. Depending on the encrypt, encryption or decryption occurs. Input is 8 bytes long and output is 8 bytes. void RC2_cbc_encrypt( unsigned char *in, unsigned char *out, long length, RC2_KEY *ks, unsigned char *ivec, int encrypt); This routine implements RC2 in Cipher Block Chaining mode. Input, which should be a multiple of 8 bytes is encrypted (or decrypted) to output which will also be a multiple of 8 bytes. The number of bytes is in length (and from what I've said above, should be a multiple of 8). If length is not a multiple of 8, bad things will probably happen. ivec is the initialisation vector. This function updates iv after each call so that it can be passed to the next call to RC2_cbc_encrypt(). void RC2_cfb64_encrypt( unsigned char *in, unsigned char *out, long length, RC2_KEY *schedule, unsigned char *ivec, int *num, int encrypt); This is one of the more useful functions in this RC2 library, it implements CFB mode of RC2 with 64bit feedback. This allows you to encrypt an arbitrary number of bytes, you do not require 8 byte padding. Each call to this routine will encrypt the input bytes to output and then update ivec and num. Num contains 'how far' we are though ivec. 'Encrypt' is used to indicate encryption or decryption. CFB64 mode operates by using the cipher to generate a stream of bytes which is used to encrypt the plain text. The cipher text is then encrypted to generate the next 64 bits to be xored (incrementally) with the next 64 bits of plain text. As can be seen from this, to encrypt or decrypt, the same 'cipher stream' needs to be generated but the way the next block of data is gathered for encryption is different for encryption and decryption. void RC2_ofb64_encrypt( unsigned char *in, unsigned char *out, long length, RC2_KEY *schedule, unsigned char *ivec, int *num); This functions implements OFB mode of RC2 with 64bit feedback. This allows you to encrypt an arbitrary number of bytes, you do not require 8 byte padding. Each call to this routine will encrypt the input bytes to output and then update ivec and num. Num contains 'how far' we are though ivec. This is in effect a stream cipher, there is no encryption or decryption mode. For reading passwords, I suggest using des_read_pw_string() from my DES library. To generate a password from a text string, I suggest using MD5 (or MD2) to produce a 16 byte message digest that can then be passed directly to RC2_set_key(). ===== For more information about the specific RC2 modes in this library (ecb, cbc, cfb and ofb), read the section entitled 'Modes of DES' from the documentation on my DES library. What is said about DES is directly applicable for RC2. ==== rc4.doc ======================================================== The RC4 library. RC4 is a stream cipher that operates on a byte stream. It can be used with any length key but I would recommend normally using 16 bytes. This library requires the inclusion of 'rc4.h'. The RC4 encryption function takes what is called an RC4_KEY as an argument. The RC4_KEY is generated by the RC4_set_key function from the key bytes. RC4, being a stream cipher, does not have an encryption or decryption mode. It produces a stream of bytes that the input stream is xor'ed against and so decryption is just a case of 'encrypting' again with the same key. I have only put in one 'mode' for RC4 which is the normal one. This means there is no initialisation vector and there is no feedback of the cipher text into the cipher. This implies that you should not ever use the same key twice if you can help it. If you do, you leave yourself open to known plain text attacks; if you know the plain text and corresponding cipher text in one message, all messages that used the same key can have the cipher text decoded for the corresponding positions in the cipher stream. The main positive feature of RC4 is that it is a very fast cipher; about 4 times faster that DES. This makes it ideally suited to protocols where the key is randomly chosen, like SSL. The functions are as follows: void RC4_set_key( RC4_KEY *key; int len; unsigned char *data); This function initialises the RC4_KEY structure with the key passed in 'data', which is 'len' bytes long. The key data can be any length but 16 bytes seems to be a good number. void RC4( RC4_KEY *key; unsigned long len; unsigned char *in; unsigned char *out); Do the actual RC4 encryption/decryption. Using the 'key', 'len' bytes are transformed from 'in' to 'out'. As mentioned above, decryption is the operation as encryption. ==== ref.doc ======================================================== I have lots more references etc, and will update this list in the future, 30 Aug 1996 - eay SSL The SSL Protocol - from Netscapes. RC4 Newsgroups: sci.crypt From: sterndark@netcom.com (David Sterndark) Subject: RC4 Algorithm revealed. Message-ID: <sternCvKL4B.Hyy@netcom.com> RC2 Newsgroups: sci.crypt From: pgut01@cs.auckland.ac.nz (Peter Gutmann) Subject: Specification for Ron Rivests Cipher No.2 Message-ID: <4fk39f$f70@net.auckland.ac.nz> MD2 RFC1319 The MD2 Message-Digest Algorithm MD5 RFC1321 The MD5 Message-Digest Algorithm X509 Certificates RFC1421 Privacy Enhancement for Internet Electronic Mail: Part I RFC1422 Privacy Enhancement for Internet Electronic Mail: Part II RFC1423 Privacy Enhancement for Internet Electronic Mail: Part III RFC1424 Privacy Enhancement for Internet Electronic Mail: Part IV RSA and various standard encoding PKCS#1 RSA Encryption Standard PKCS#5 Password-Based Encryption Standard PKCS#7 Cryptographic Message Syntax Standard A Layman's Guide to a Subset of ASN.1, BER, and DER An Overview of the PKCS Standards Some Examples of the PKCS Standards IDEA Chapter 3 The Block Cipher IDEA RSA, prime number generation and bignum algorithms Introduction To Algorithms, Thomas Cormen, Charles Leiserson, Ronald Rivest, Section 29 Arithmetic Circuits Section 33 Number-Theoretic Algorithms Fast Private Key algorithm Fast Decipherment Algorithm for RSA Public-Key Cryptosystem J.-J. Quisquater and C. Couvreur, Electronics Letters, 14th October 1982, Vol. 18 No. 21 Prime number generation and bignum algorithms. PGP-2.3a ==== rsa.doc ======================================================== The RSA encryption and utility routines. The RSA routines are built on top of a big number library (the BN library). There are support routines in the X509 library for loading and manipulating the various objects in the RSA library. When errors are returned, read about the ERR library for how to access the error codes. All RSA encryption is done according to the PKCS-1 standard which is compatible with PEM and RSAref. This means that any values being encrypted must be less than the size of the modulus in bytes, minus 10, bytes long. This library uses RAND_bytes()() for it's random data, make sure to feed RAND_seed() with lots of interesting and varied data before using these routines. The RSA library has one specific data type, the RSA structure. It is composed of 8 BIGNUM variables (see the BN library for details) and can hold either a private RSA key or a public RSA key. Some RSA libraries have different structures for public and private keys, I don't. For my libraries, a public key is determined by the fact that the RSA->d value is NULL. These routines will operate on any size RSA keys. While I'm sure 4096 bit keys are very very secure, they take a lot longer to process that 1024 bit keys :-). The function in the RSA library are as follows. RSA *RSA_new(); This function creates a new RSA object. The sub-fields of the RSA type are also malloced so you should always use this routine to create RSA variables. void RSA_free( RSA *rsa); This function 'frees' an RSA structure. This routine should always be used to free the RSA structure since it will also 'free' any sub-fields of the RSA type that need freeing. int RSA_size( RSA *rsa); This function returns the size of the RSA modulus in bytes. Why do I need this you may ask, well the reason is that when you encrypt with RSA, the output string will be the size of the RSA modulus. So the output for the RSA_encrypt and the input for the RSA_decrypt routines need to be RSA_size() bytes long, because this is how many bytes are expected. For the following 4 RSA encryption routines, it should be noted that RSA_private_decrypt() should be used on the output from RSA_public_encrypt() and RSA_public_decrypt() should be used on the output from RSA_private_encrypt(). int RSA_public_encrypt( int from_len; unsigned char *from unsigned char *to RSA *rsa); This function implements RSA public encryption, the rsa variable should be a public key (but can be a private key). 'from_len' bytes taken from 'from' and encrypted and put into 'to'. 'to' needs to be at least RSA_size(rsa) bytes long. The number of bytes written into 'to' is returned. -1 is returned on an error. The operation performed is to = from^rsa->e mod rsa->n. int RSA_private_encrypt( int from_len; unsigned char *from unsigned char *to RSA *rsa); This function implements RSA private encryption, the rsa variable should be a private key. 'from_len' bytes taken from 'from' and encrypted and put into 'to'. 'to' needs to be at least RSA_size(rsa) bytes long. The number of bytes written into 'to' is returned. -1 is returned on an error. The operation performed is to = from^rsa->d mod rsa->n. int RSA_public_decrypt( int from_len; unsigned char *from unsigned char *to RSA *rsa); This function implements RSA public decryption, the rsa variable should be a public key (but can be a private key). 'from_len' bytes are taken from 'from' and decrypted. The decrypted data is put into 'to'. The number of bytes encrypted is returned. -1 is returned to indicate an error. The operation performed is to = from^rsa->e mod rsa->n. int RSA_private_decrypt( int from_len; unsigned char *from unsigned char *to RSA *rsa); This function implements RSA private decryption, the rsa variable should be a private key. 'from_len' bytes are taken from 'from' and decrypted. The decrypted data is put into 'to'. The number of bytes encrypted is returned. -1 is returned to indicate an error. The operation performed is to = from^rsa->d mod rsa->n. int RSA_mod_exp( BIGNUM *n; BIGNUM *p; RSA *rsa); Normally you will never use this routine. This is really an internal function which is called by RSA_private_encrypt() and RSA_private_decrypt(). It performs n=n^p mod rsa->n except that it uses the 5 extra variables in the RSA structure to make this more efficient. RSA *RSA_generate_key( int bits; unsigned long e; void (*callback)(); char *cb_arg; This routine is used to generate RSA private keys. It takes quite a period of time to run and should only be used to generate initial private keys that should then be stored for later use. The passed callback function will be called periodically so that feedback can be given as to how this function is progressing. 'bits' is the length desired for the modulus, so it would be 1024 to generate a 1024 bit private key. 'e' is the value to use for the public exponent 'e'. Traditionally it is set to either 3 or 0x10001. The callback function (if not NULL) is called in the following situations. when we have generated a suspected prime number to test, callback(0,num1++,cb_arg). When it passes a prime number test, callback(1,num2++,cb_arg). When it is rejected as one of the 2 primes required due to gcd(prime,e value) != 0, callback(2,num3++,cb_arg). When finally accepted as one of the 2 primes, callback(3,num4++,cb_arg). ==== rsaref.doc ======================================================== This package can be compiled to use the RSAref library. This library is not allowed outside of the USA but inside the USA it is claimed by RSA to be the only RSA public key library that can be used besides BSAFE.. There are 2 files, rsaref/rsaref.c and rsaref/rsaref.h that contain the glue code to use RSAref. These files were written by looking at the PGP source code and seeing which routines it used to access RSAref. I have also been sent by some-one a copy of the RSAref header file that contains the library error codes. [ Jun 1996 update - I have recently gotten hold of RSAref 2.0 from South Africa and have been doing some performace tests. ] They have now been tested against the recently announced RSAEURO library. There are 2 ways to use SSLeay and RSAref. First, to build so that the programs must be linked with RSAref, add '-DRSAref' to CFLAG in the top level makefile and -lrsaref (or where ever you are keeping RSAref) to EX_LIBS. To build a makefile via util/mk1mf.pl to do this, use the 'rsaref' option. The second method is to build as per normal and link applications with the RSAglue library. The correct library order would be cc -o cmd cmd.o -lssl -lRSAglue -lcrypto -lrsaref -ldes The RSAglue library is built in the rsa directory and is NOT automatically installed. Be warned that the RSAEURO library, that is claimed to be compatible with RSAref contains a different value for the maximum number of bits supported. This changes structure sizes and so if you are using RSAEURO, change the value of RSAref_MAX_BITS in rsa/rsaref.h ==== s_mult.doc ======================================================== s_mult is a test program I hacked up on a Sunday for testing non-blocking IO. It has a select loop at it's centre that handles multiple readers and writers. Try the following command ssleay s_mult -echo -nbio -ssl -v echo - sends any sent text back to the sender nbio - turns on non-blocking IO ssl - accept SSL connections, default is normal text v - print lots type Q<cr> to quit In another window, run the following ssleay s_client -pause </etc/termcap The pause option puts in a 1 second pause in each read(2)/write(2) call so the other end will have read()s fail. ==== session.doc ======================================================== I have just checked over and re-worked the session stuff. The following brief example will ignore all setup information to do with authentication. Things operate as follows. The SSL environment has a 'context', a SSL_CTX structure. This holds the cached SSL_SESSIONS (which can be reused) and the certificate lookup information. Each SSL structure needs to be associated with a SSL_CTX. Normally only one SSL_CTX structure is needed per program. SSL_CTX *SSL_CTX_new(void ); void SSL_CTX_free(SSL_CTX *); These 2 functions create and destroy SSL_CTX structures The SSL_CTX has a session_cache_mode which is by default, in SSL_SESS_CACHE_SERVER mode. What this means is that the library will automatically add new session-id's to the cache upon successful SSL_accept() calls. If SSL_SESS_CACHE_CLIENT is set, then client certificates are also added to the cache. SSL_set_session_cache_mode(ctx,mode) will set the 'mode' and SSL_get_session_cache_mode(ctx) will get the cache 'mode'. The modes can be SSL_SESS_CACHE_OFF - no caching SSL_SESS_CACHE_CLIENT - only SSL_connect() SSL_SESS_CACHE_SERVER - only SSL_accept() SSL_SESS_NO_CACHE_BOTH - Either SSL_accept() or SSL_connect(). If SSL_SESS_CACHE_NO_AUTO_CLEAR is set, old timed out sessions are not automatically removed each 255, SSL_connect()s or SSL_accept()s. By default, upon every 255 successful SSL_connect() or SSL_accept()s, the cache is flush. Please note that this could be expensive on a heavily loaded SSL server, in which case, turn this off and clear the cache of old entries 'manually' (with one of the functions listed below) every few hours. Perhaps I should up this number, it is hard to say. Remember, the '255' new calls is just a mechanism to get called every now and then, in theory at most 255 new session-id's will have been added but if 100 are added every minute, you would still have 500 in the cache before any would start being flushed (assuming a 3 minute timeout).. int SSL_CTX_sess_hits(SSL_CTX *ctx); int SSL_CTX_sess_misses(SSL_CTX *ctx); int SSL_CTX_sess_timeouts(SSL_CTX *ctx); These 3 functions return statistics about the SSL_CTX. These 3 are the number of session id reuses. hits is the number of reuses, misses are the number of lookups that failed, and timeouts is the number of cached entries ignored because they had timeouted. ctx->new_session_cb is a function pointer to a function of type int new_session_callback(SSL *ssl,SSL_SESSION *new); This function, if set in the SSL_CTX structure is called whenever a new SSL_SESSION is added to the cache. If the callback returns non-zero, it means that the application will have to do a SSL_SESSION_free() on the structure (this is to do with the cache keeping the reference counts correct, without the application needing to know about it. The 'active' parameter is the current SSL session for which this connection was created. void SSL_CTX_sess_set_new_cb(SSL_CTX *ctx,int (*cb)()); to set the callback, int (*cb)() SSL_CTX_sess_get_new_cb(SSL_CTX *ctx) to get the callback. If the 'get session' callback is set, when a session id is looked up and it is not in the session-id cache, this callback is called. The callback is of the form SSL_SESSION *get_session_callback(unsigned char *sess_id,int sess_id_len, int *copy); The get_session_callback is intended to return null if no session id is found. The reference count on the SSL_SESSION in incremented by the SSL library, if copy is 1. Otherwise, the reference count is not modified. void SSL_CTX_sess_set_get_cb(ctx,cb) sets the callback and int (*cb)()SSL_CTX_sess_get_get_cb(ctx) returns the callback. These callbacks are basically intended to be used by processes to send their session-id's to other processes. I currently have not implemented non-blocking semantics for these callbacks, it is upto the application to make the callbacks efficient if they require blocking (perhaps by 'saving' them and then 'posting them' when control returns from the SSL_accept(). LHASH *SSL_CTX_sessions(SSL_CTX *ctx) This returns the session cache. The lhash strucutre can be accessed for statistics about the cache. void lh_stats(LHASH *lh, FILE *out); void lh_node_stats(LHASH *lh, FILE *out); void lh_node_usage_stats(LHASH *lh, FILE *out); can be used to print details about it's activity and current state. You can also delve directly into the lhash structure for 14 different counters that are kept against the structure. When I wrote the lhash library, I was interested in gathering statistics :-). Have a read of doc/lhash.doc in the SSLeay distribution area for more details on the lhash library. Now as mentioned ealier, when a SSL is created, it needs a SSL_CTX. SSL * SSL_new(SSL_CTX *); This stores a session. A session is secret information shared between 2 SSL contexts. It will only be created if both ends of the connection have authenticated their peer to their satisfaction. It basically contains the information required to use a particular secret key cipher. To retrieve the SSL_CTX being used by a SSL, SSL_CTX *SSL_get_SSL_CTX(SSL *s); Now when a SSL session is established between to programs, the 'session' information that is cached in the SSL_CTX can me manipulated by the following functions. int SSL_set_session(SSL *s, SSL_SESSION *session); This will set the SSL_SESSION to use for the next SSL_connect(). If you use this function on an already 'open' established SSL connection, 'bad things will happen'. This function is meaning-less when used on a ssl strucutre that is just about to be used in a SSL_accept() call since the SSL_accept() will either create a new session or retrieve one from the cache. SSL_SESSION *SSL_get_session(SSL *s); This will return the SSL_SESSION for the current SSL, NULL if there is no session associated with the SSL structure. The SSL sessions are kept in the SSL_CTX in a hash table, to remove a session void SSL_CTX_remove_session(SSL_CTX *,SSL_SESSION *c); and to add one int SSL_CTX_add_session(SSL_CTX *s, SSL_SESSION *c); SSL_CTX_add_session() returns 1 if the session was already in the cache (so it was not added). Whenever a new session is created via SSL_connect()/SSL_accept(), they are automatically added to the cache, depending on the session_cache_mode settings. SSL_set_session() does not add it to the cache. Just call SSL_CTX_add_session() if you do want the session added. For a 'client' this would not normally be the case. SSL_CTX_add_session() is not normally ever used, except for doing 'evil' things which the next 2 funtions help you do. int i2d_SSL_SESSION(SSL_SESSION *in,unsigned char **pp); SSL_SESSION *d2i_SSL_SESSION(SSL_SESSION **a,unsigned char **pp,long length); These 2 functions are in the standard ASN1 library form and can be used to load and save to a byte format, the SSL_SESSION structure. With these functions, you can save and read these structures to a files or arbitary byte string. The PEM_write_SSL_SESSION(fp,x) and PEM_read_SSL_SESSION(fp,x,cb) will write to a file pointer in base64 encoding. What you can do with this, is pass session information between separate processes. Please note, that you will probably also need to modify the timeout information on the SSL_SESSIONs. long SSL_get_time(SSL_SESSION *s) will return the 'time' that the session was loaded. The timeout is relative to this time. This information is saved when the SSL_SESSION is converted to binarary but it is stored in as a unix long, which is rather OS dependant, but easy to convert back. long SSL_set_time(SSL_SESSION *s,long t) will set the above mentioned time. The time value is just the value returned from time(3), and should really be defined by be to be time_t. long SSL_get_timeout(SSL_SESSION *s); long SSL_set_timeout(SSL_SESSION *s,long t); These 2 retrieve and set the timeout which is just a number of secconds from the 'SSL_get_time()' value. When this time period has elapesed, the session will no longer be in the cache (well it will actually be removed the next time it is attempted to be retrieved, so you could 'bump' the timeout so it remains valid). The 'time' and 'timeout' are set on a session when it is created, not reset each time it is reused. If you did wish to 'bump it', just after establishing a connection, do a SSL_set_time(ssl,time(NULL)); You can also use SSL_CTX_set_timeout(SSL_CTX *ctx,unsigned long t) and SSL_CTX_get_timeout(SSL_CTX *ctx) to manipulate the default timeouts for all SSL connections created against a SSL_CTX. If you set a timeout in an SSL_CTX, all new SSL's created will inherit the timeout. It can be over written by the SSL_set_timeout(SSL *s,unsigned long t) function call. If you 'set' the timeout back to 0, the system default will be used. SSL_SESSION *SSL_SESSION_new(); void SSL_SESSION_free(SSL_SESSION *ses); These 2 functions are used to create and dispose of SSL_SESSION functions. You should not ever normally need to use them unless you are using i2d_SSL_SESSION() and/or d2i_SSL_SESSION(). If you 'load' a SSL_SESSION via d2i_SSL_SESSION(), you will need to SSL_SESSION_free() it. Both SSL_set_session() and SSL_CTX_add_session() will 'take copies' of the structure (via reference counts) when it is passed to them. SSL_CTX_flush_sessions(ctx,time); The first function will clear all sessions from the cache, which have expired relative to 'time' (which could just be time(NULL)). SSL_CTX_flush_sessions(ctx,0); This is a special case that clears everything. As a final comment, a 'session' is not enough to establish a new connection. If a session has timed out, a certificate and private key need to have been associated with the SSL structure. SSL_copy_session_id(SSL *to,SSL *from); will copy not only the session strucutre but also the private key and certificate associated with 'from'. EXAMPLES. So lets play at being a weird SSL server. /* setup a context */ ctx=SSL_CTX_new(); /* Lets load some session from binary into the cache, why one would do * this is not toally clear, but passing between programs does make sense * Perhaps you are using 4096 bit keys and are happy to keep them * valid for a week, to avoid the RSA overhead of 15 seconds, I'm not toally * sure, perhaps this is a process called from an SSL inetd and this is being * passed to the application. */ session=d2i_SSL_SESSION(....) SSL_CTX_add_session(ctx,session); /* Lets even add a session from a file */ session=PEM_read_SSL_SESSION(....) SSL_CTX_add_session(ctx,session); /* create a new SSL structure */ ssl=SSL_new(ctx); /* At this point we want to be able to 'create' new session if * required, so we need a certificate and RSAkey. */ SSL_use_RSAPrivateKey_file(ssl,...) SSL_use_certificate_file(ssl,...) /* Now since we are a server, it make little sence to load a session against * the ssl strucutre since a SSL_accept() will either create a new session or * grab an existing one from the cache. */ /* grab a socket descriptor */ fd=accept(...); /* associated it with the ssl strucutre */ SSL_set_fd(ssl,fd); SSL_accept(ssl); /* 'do' SSL using out cert and RSA key */ /* Lets print out the session details or lets save it to a file, * perhaps with a secret key cipher, so that we can pass it to the FBI * when they want to decode the session :-). While we have RSA * this does not matter much but when I do SSLv3, this will allow a mechanism * for the server/client to record the information needed to decode * the traffic that went over the wire, even when using Diffie-Hellman */ PEM_write_SSL_SESSION(SSL_get_session(ssl),stdout,....) Lets 'connect' back to the caller using the same session id. ssl2=SSL_new(ctx); fd2=connect(them); SSL_set_fd(ssl2,fd2); SSL_set_session(ssl2,SSL_get_session(ssl)); SSL_connect(ssl2); /* what the hell, lets accept no more connections using this session */ SSL_CTX_remove_session(SSL_get_SSL_CTX(ssl),SSL_get_session(ssl)); /* we could have just as easily used ssl2 since they both are using the * same session. * You will note that both ssl and ssl2 are still using the session, and * the SSL_SESSION structure will be free()ed when both ssl and ssl2 * finish using the session. Also note that you could continue to initiate * connections using this session by doing SSL_get_session(ssl) to get the * existing session, but SSL_accept() will not be able to find it to * use for incoming connections. * Of corse, the session will timeout at the far end and it will no * longer be accepted after a while. The time and timeout are ignored except * by SSL_accept(). */ /* Since we have had our server running for 10 weeks, and memory is getting * short, perhaps we should clear the session cache to remove those * 100000 session entries that have expired. Some may consider this * a memory leak :-) */ SSL_CTX_flush_sessions(ctx,time(NULL)); /* Ok, after a bit more time we wish to flush all sessions from the cache * so that all new connections will be authenticated and incure the * public key operation overhead */ SSL_CTX_flush_sessions(ctx,0); /* As a final note, to copy everything to do with a SSL, use */ SSL_copy_session_id(SSL *to,SSL *from); /* as this also copies the certificate and RSA key so new session can * be established using the same details */ ==== sha.doc ======================================================== The SHA (Secure Hash Algorithm) library. SHA is a message digest algorithm that can be used to condense an arbitrary length message down to a 20 byte hash. The functions all need to be passed a SHA_CTX which is used to hold the SHA context during multiple SHA_Update() function calls. The normal method of use for this library is as follows This library contains both SHA and SHA-1 digest algorithms. SHA-1 is an update to SHA (which should really be called SHA-0 now) which tweaks the algorithm slightly. The SHA-1 algorithm is used by simply using SHA1_Init(), SHA1_Update(), SHA1_Final() and SHA1() instead of the SHA*() calls SHA_Init(...); SHA_Update(...); ... SHA_Update(...); SHA_Final(...); This library requires the inclusion of 'sha.h'. The functions are as follows: void SHA_Init( SHA_CTX *c); This function needs to be called to initiate a SHA_CTX structure for use. void SHA_Update( SHA_CTX *c; unsigned char *data; unsigned long len); This updates the message digest context being generated with 'len' bytes from the 'data' pointer. The number of bytes can be any length. void SHA_Final( unsigned char *md; SHA_CTX *c; This function is called when a message digest of the data digested with SHA_Update() is wanted. The message digest is put in the 'md' array and is SHA_DIGEST_LENGTH (20) bytes long. unsigned char *SHA( unsigned char *d; unsigned long n; unsigned char *md; This function performs a SHA_Init(), followed by a SHA_Update() followed by a SHA_Final() (using a local SHA_CTX). The resulting digest is put into 'md' if it is not NULL. Regardless of the value of 'md', the message digest is returned from the function. If 'md' was NULL, the message digest returned is being stored in a static structure. ==== speed.doc ======================================================== To get an idea of the performance of this library, use ssleay speed perl util/sp-diff.pl file1 file2 will print out the relative differences between the 2 files which are expected to be the output from the speed program. The performace of the library is very dependant on the Compiler quality and various flags used to build. --- These are some numbers I did comparing RSAref and SSLeay on a Pentium 100. [ These numbers are all out of date, as of SSL - 0.6.1 the RSA operations are about 2 times faster, so check the version number ] RSA performance. SSLeay 0.6.0 Pentium 100, 32meg, Windows NT Workstation 3.51 linux - gcc v 2.7.0 -O3 -fomit-frame-pointer -m486 and Windows NT - Windows NT 3.51 - Visual C++ 4.1 - 586 code + 32bit assember Windows 3.1 - Windows NT 3.51 - Visual C++ 1.52c - 286 code + 32bit assember NT Dos Shell- Windows NT 3.51 - Visual C++ 1.52c - 286 code + 16bit assember Times are how long it takes to do an RSA private key operation. 512bits 1024bits ------------------------------- SSLeay NT dll 0.042s 0.202s see above SSLeay linux 0.046s 0.218s Assember inner loops (normal build) SSLeay linux 0.067s 0.380s Pure C code with BN_LLONG defined SSLeay W3.1 dll 0.108s 0.478s see above SSLeay linux 0.109s 0.713s C without BN_LLONG. RSAref2.0 linux 0.149s 0.936s SSLeay MS-DOS 0.197s 1.049s see above 486DX66, 32meg, Windows NT Server 3.51 512bits 1024bits ------------------------------- SSLeay NT dll 0.084s 0.495s <- SSLeay 0.6.3 SSLeay NT dll 0.154s 0.882s SSLeay W3.1 dll 0.335s 1.538s SSLeay MS-DOS 0.490s 2.790s What I find cute is that I'm still faster than RSAref when using standard C, without using the 'long long' data type :-), %35 faster for 512bit and we scale up to 3.2 times faster for the 'default linux' build. I should mention that people should 'try' to use either x86-lnx.s (elf), x86-lnxa.s or x86-sol.s for any x86 based unix they are building on. The only problems with be with syntax but the performance gain is quite large, especially for servers. The code is very simple, you just need to modify the 'header'. The message is, if you are stuck using RSAref, the RSA performance will be bad. Considering the code was compiled for a pentium, the 486DX66 number would indicate 'Use RSAref and turn you Pentium 100 into a 486DX66' :-). [ As of verson 0.6.1, it would be correct to say 'turn you pentium 100 into a 486DX33' :-) ] I won't tell people if the DLL's are using RSAref or my stuff if no-one asks :-). eric PS while I know I could speed things up further, I will probably not do so due to the effort involved. I did do some timings on the SSLeay bignum format -> RSAref number format conversion that occurs each time RSAref is used by SSLeay, and the numbers are trivial. 0.00012s a call for 512bit vs 0.149s for the time spent in the function. 0.00018s for 1024bit vs 0.938s. Insignificant. So the 'way to go', to support faster RSA libraries, if people are keen, is to write 'glue' code in a similar way that I do for RSAref and send it to me :-). My base library still has the advantage of being able to operate on any size numbers, and is not that far from the performance from the leaders in the field. (-%30?) [ Well as of 0.6.1 I am now the leader in the filed on x86 (we at least very close :-) ] I suppose I should also mention some other numbers RSAref numbers, again on my Pentium. DES CBC EDE-DES MD5 RSAref linux 830k/s 302k/s 4390k/s SSLeay linux 855k/s 319k/s 10025k/s SSLeay NT 1158k/s 410k/s 10470k/s SSLeay w31 378k/s 143k/s 2383k/s (fully 16bit) Got to admit that Visual C++ 4.[01] is a damn fine compiler :-) -- Eric Young | BOOL is tri-state according to Bill Gates. AARNet: eay@cryptsoft.com | RTFM Win32 GetMessage(). ==== ssl-ciph.doc ======================================================== This is a quick high level summery of how things work now. Each SSLv2 and SSLv3 cipher is composed of 4 major attributes plus a few extra minor ones. They are 'The key exchange algorithm', which is RSA for SSLv2 but can also be Diffle-Hellman for SSLv3. An 'Authenticion algorithm', which can be RSA, Diffle-Helman, DSS or none. The cipher The MAC digest. A cipher can also be an export cipher and is either an SSLv2 or a SSLv3 ciphers. To specify which ciphers to use, one can either specify all the ciphers, one at a time, or use 'aliases' to specify the preference and order for the ciphers. There are a large number of aliases, but the most importaint are kRSA, kDHr, kDHd and kEDH for key exchange types. aRSA, aDSS, aNULL and aDH for authentication DES, 3DES, RC4, RC2, IDEA and eNULL for ciphers MD5, SHA0 and SHA1 digests Now where this becomes interesting is that these can be put together to specify the order and ciphers you wish to use. To speed this up there are also aliases for certian groups of ciphers. The main ones are SSLv2 - all SSLv2 ciphers SSLv3 - all SSLv3 ciphers EXP - all export ciphers LOW - all low strngth ciphers (no export ciphers, normally single DES) MEDIUM - 128 bit encryption HIGH - Triple DES These aliases can be joined in a : separated list which specifies to add ciphers, move them to the current location and delete them. A simpler way to look at all of this is to use the 'ssleay ciphers -v' command. The default library cipher spec is !ADH:RC4+RSA:HIGH:MEDIUM:LOW:EXP:+SSLv2:+EXP which means, first, remove from consideration any ciphers that do not authenticate. Next up, use ciphers using RC4 and RSA. Next include the HIGH, MEDIUM and the LOW security ciphers. Finish up by adding all the export ciphers on the end, then 'pull' all the SSLv2 and export ciphers to the end of the list. The results are $ ssleay ciphers -v '!ADH:RC4+RSA:HIGH:MEDIUM:LOW:EXP:+SSLv2:+EXP' RC4-SHA SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=SHA1 RC4-MD5 SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5 EDH-RSA-DES-CBC3-SHA SSLv3 Kx=DH Au=RSA Enc=3DES(168) Mac=SHA1 EDH-DSS-DES-CBC3-SHA SSLv3 Kx=DH Au=DSS Enc=3DES(168) Mac=SHA1 DES-CBC3-SHA SSLv3 Kx=RSA Au=RSA Enc=3DES(168) Mac=SHA1 IDEA-CBC-MD5 SSLv3 Kx=RSA Au=RSA Enc=IDEA(128) Mac=SHA1 EDH-RSA-DES-CBC-SHA SSLv3 Kx=DH Au=RSA Enc=DES(56) Mac=SHA1 EDH-DSS-DES-CBC-SHA SSLv3 Kx=DH Au=DSS Enc=DES(56) Mac=SHA1 DES-CBC-SHA SSLv3 Kx=RSA Au=RSA Enc=DES(56) Mac=SHA1 DES-CBC3-MD5 SSLv2 Kx=RSA Au=RSA Enc=3DES(168) Mac=MD5 DES-CBC-MD5 SSLv2 Kx=RSA Au=RSA Enc=DES(56) Mac=MD5 IDEA-CBC-MD5 SSLv2 Kx=RSA Au=RSA Enc=IDEA(128) Mac=MD5 RC2-CBC-MD5 SSLv2 Kx=RSA Au=RSA Enc=RC2(128) Mac=MD5 RC4-MD5 SSLv2 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5 EXP-EDH-RSA-DES-CBC SSLv3 Kx=DH(512) Au=RSA Enc=DES(40) Mac=SHA1 export EXP-EDH-DSS-DES-CBC-SHA SSLv3 Kx=DH(512) Au=DSS Enc=DES(40) Mac=SHA1 export EXP-DES-CBC-SHA SSLv3 Kx=RSA(512) Au=RSA Enc=DES(40) Mac=SHA1 export EXP-RC2-CBC-MD5 SSLv3 Kx=RSA(512) Au=RSA Enc=RC2(40) Mac=MD5 export EXP-RC4-MD5 SSLv3 Kx=RSA(512) Au=RSA Enc=RC4(40) Mac=MD5 export EXP-RC2-CBC-MD5 SSLv2 Kx=RSA(512) Au=RSA Enc=RC2(40) Mac=MD5 export EXP-RC4-MD5 SSLv2 Kx=RSA(512) Au=RSA Enc=RC4(40) Mac=MD5 export I would recoment people use the 'ssleay ciphers -v "text"' command to check what they are going to use. Anyway, I'm falling asleep here so I'll do some more tomorrow. eric ==== ssl.doc ======================================================== SSL_CTX_sessions(SSL_CTX *ctx) - the session-id hash table. /* Session-id cache stats */ SSL_CTX_sess_number SSL_CTX_sess_connect SSL_CTX_sess_connect_good SSL_CTX_sess_accept SSL_CTX_sess_accept_good SSL_CTX_sess_hits SSL_CTX_sess_cb_hits SSL_CTX_sess_misses SSL_CTX_sess_timeouts /* Session-id application notification callbacks */ SSL_CTX_sess_set_new_cb SSL_CTX_sess_get_new_cb SSL_CTX_sess_set_get_cb SSL_CTX_sess_get_get_cb /* Session-id cache operation mode */ SSL_CTX_set_session_cache_mode SSL_CTX_get_session_cache_mode /* Set default timeout values to use. */ SSL_CTX_set_timeout SSL_CTX_get_timeout /* Global SSL initalisation informational callback */ SSL_CTX_set_info_callback SSL_CTX_get_info_callback SSL_set_info_callback SSL_get_info_callback /* If the SSL_accept/SSL_connect returned with -1, these indicate when * we should re-call *. SSL_want SSL_want_nothing SSL_want_read SSL_want_write SSL_want_x509_lookup /* Where we are in SSL initalisation, used in non-blocking, perhaps * have a look at ssl/bio_ssl.c */ SSL_state SSL_is_init_finished SSL_in_init SSL_in_connect_init SSL_in_accept_init /* Used to set the 'inital' state so SSL_in_connect_init and SSL_in_accept_init * can be used to work out which function to call. */ SSL_set_connect_state SSL_set_accept_state /* Where to look for certificates for authentication */ SSL_set_default_verify_paths /* calles SSL_load_verify_locations */ SSL_load_verify_locations /* get info from an established connection */ SSL_get_session SSL_get_certificate SSL_get_SSL_CTX SSL_CTX_new SSL_CTX_free SSL_new SSL_clear SSL_free SSL_CTX_set_cipher_list SSL_get_cipher SSL_set_cipher_list SSL_get_cipher_list SSL_get_shared_ciphers SSL_accept SSL_connect SSL_read SSL_write SSL_debug SSL_get_read_ahead SSL_set_read_ahead SSL_set_verify SSL_pending SSL_set_fd SSL_set_rfd SSL_set_wfd SSL_set_bio SSL_get_fd SSL_get_rbio SSL_get_wbio SSL_use_RSAPrivateKey SSL_use_RSAPrivateKey_ASN1 SSL_use_RSAPrivateKey_file SSL_use_PrivateKey SSL_use_PrivateKey_ASN1 SSL_use_PrivateKey_file SSL_use_certificate SSL_use_certificate_ASN1 SSL_use_certificate_file ERR_load_SSL_strings SSL_load_error_strings /* human readable version of the 'state' of the SSL connection. */ SSL_state_string SSL_state_string_long /* These 2 report what kind of IO operation the library was trying to * perform last. Probably not very usefull. */ SSL_rstate_string SSL_rstate_string_long SSL_get_peer_certificate SSL_SESSION_new SSL_SESSION_print_fp SSL_SESSION_print SSL_SESSION_free i2d_SSL_SESSION d2i_SSL_SESSION SSL_get_time SSL_set_time SSL_get_timeout SSL_set_timeout SSL_copy_session_id SSL_set_session SSL_CTX_add_session SSL_CTX_remove_session SSL_CTX_flush_sessions BIO_f_ssl /* used to hold information as to why a certificate verification failed */ SSL_set_verify_result SSL_get_verify_result /* can be used by the application to associate data with an SSL structure. * It needs to be 'free()ed' by the application */ SSL_set_app_data SSL_get_app_data /* The following all set values that are kept in the SSL_CTX but * are used as the default values when an SSL session is created. * They are over writen by the relevent SSL_xxxx functions */ /* SSL_set_verify */ void SSL_CTX_set_default_verify /* This callback, if set, totaly overrides the normal SSLeay verification * functions and should return 1 on success and 0 on failure */ void SSL_CTX_set_cert_verify_callback /* The following are the same as the equivilent SSL_xxx functions. * Only one copy of this information is kept and if a particular * SSL structure has a local override, it is totally separate structure. */ int SSL_CTX_use_RSAPrivateKey int SSL_CTX_use_RSAPrivateKey_ASN1 int SSL_CTX_use_RSAPrivateKey_file int SSL_CTX_use_PrivateKey int SSL_CTX_use_PrivateKey_ASN1 int SSL_CTX_use_PrivateKey_file int SSL_CTX_use_certificate int SSL_CTX_use_certificate_ASN1 int SSL_CTX_use_certificate_file ==== ssl_ctx.doc ======================================================== This is now a bit dated, quite a few of the SSL_ functions could be SSL_CTX_ functions. I will update this in the future. 30 Aug 1996 From eay@orb.mincom.oz.au Mon Dec 11 21:37:08 1995 Received: by orb.mincom.oz.au id AA00696 (5.65c/IDA-1.4.4 for eay); Mon, 11 Dec 1995 11:37:08 +1000 Date: Mon, 11 Dec 1995 11:37:08 +1000 (EST) From: Eric Young <eay@mincom.oz.au> X-Sender: eay@orb To: sameer <sameer@c2.org> Cc: Eric Young <eay@mincom.oz.au> Subject: Re: PEM_readX509 oesn't seem to be working In-Reply-To: <199512110102.RAA12521@infinity.c2.org> Message-Id: <Pine.SOL.3.91.951211112115.28608D-100000@orb> Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Status: RO X-Status: On Sun, 10 Dec 1995, sameer wrote: > OK, that's solved. I've found out that it is saying "no > certificate set" in SSL_accept because s->conn == NULL > so there is some place I need to initialize s->conn that I am > not initializing it. The full order of things for a server should be. ctx=SSL_CTX_new(); /* The next line should not really be using ctx->cert but I'll leave it * this way right now... I don't want a X509_ routine to know about an SSL * structure, there should be an SSL_load_verify_locations... hmm, I may * add it tonight. */ X509_load_verify_locations(ctx->cert,CAfile,CApath); /* Ok now for each new connection we do the following */ con=SSL_new(ctx); SSL_set_fd(con,s); SSL_set_verify(con,verify,verify_callback); /* set the certificate and private key to use. */ SSL_use_certificate_ASN1(con,X509_certificate); SSL_use_RSAPrivateKey_ASN1(con,RSA_private_key); SSL_accept(con); SSL_read(con)/SSL_write(con); There is a bit more than that but that is basically the structure. Create a context and specify where to lookup certificates. foreach connection { create a SSL structure set the certificate and private key do a SSL_accept we should now be ok } eric -- Eric Young | Signature removed since it was generating AARNet: eay@mincom.oz.au | more followups than the message contents :-) ==== ssleay.doc ======================================================== SSLeay: a cryptographic kitchen sink. 1st December 1995 Way back at the start of April 1995, I was looking for a mindless programming project. A friend of mine (Tim Hudson) said "why don't you do SSL, it has DES encryption in it and I would not mind using it in a SSL telnet". While it was true I had written a DES library in previous years, litle did I know what an expansive task SSL would turn into. First of all, the SSL protocol contains DES encryption. Well and good. My DES library was fast and portable. It also contained the RSA's RC4 stream cipher. Again, not a problem, some-one had just posted to sci.crypt something that was claimed to be RC4. It also contained IDEA, I had the specifications, not a problem to implement. MD5, an RFC, trivial, at most I could spend a week or so trying to see if I could speed up the implementation. All in all a nice set of ciphers. Then the first 'expantion of the scope', RSA public key encryption. Since I did not knowing a thing about public key encryption or number theory, this appeared quite a daunting task. Just writing a big number library would be problomatic in itself, let alone making it fast. At this point the scope of 'implementing SSL' expands eponentialy. First of all, the RSA private keys were being kept in ASN.1 format. Thankfully the RSA PKCS series of documents explains this format. So I now needed to be able to encode and decode arbitary ASN.1 objects. The Public keys were embeded in X509 certificates. Hmm... these are not only ASN.1 objects but they make up a heirachy of authentication. To authenticate a X509 certificate one needs to retrieve it's issuers certificate etc etc. Hmm..., so I also need to implement some kind of certificate management software. I would also have to implement software to authenticate certificates. At this point the support code made the SSL part of my library look quite small. Around this time, the first version of SSLeay was released. Ah, but here was the problem, I was not happy with the code so far. As may have become obvious, I had been treating all of this as a learning exersize, so I have completely written the library myself. As such, due to the way it had grown like a fungus, much of the library was not 'elagent' or neat. There were global and static variables all over the place, the SSL part did not even handle non-blocking IO. The Great rewrite began. As of this point in time, the 'Great rewrite' has almost finished. So what follows is an approximate list of what is actually SSLeay 0.5.0 /********* This needs to be updated for 0.6.0+ *************/ --- The library contains the following routines. Please note that most of these functions are not specfic for SSL or any other particular cipher implementation. I have tried to make all the routines as general purpose as possible. So you should not think of this library as an SSL implemtation, but rather as a library of cryptographic functions that also contains SSL. I refer to each of these function groupings as libraries since they are often capable of functioning as independant libraries First up, the general ciphers and message digests supported by the library. MD2 rfc???, a standard 'by parts' interface to this algorithm. MD5 rfc???, the same type of interface as for the MD2 library except a different algorithm. SHA THe Secure Hash Algorithm. Again the same type of interface as MD2/MD5 except the digest is 20 bytes. SHA1 The 'revised' version of SHA. Just about identical to SHA except for one tweak of an inner loop. DES This is my libdes library that has been floating around for the last few years. It has been enhanced for no other reason than completeness. It now supports ecb, cbc, cfb, ofb, cfb64, ofb64 in normal mode and triple DES modes of ecb, cbc, cfb64 and ofb64. cfb64 and ofb64 are functional interfaces to the 64 bit modes of cfb and ofb used in such a way thay they function as single character interfaces. RC4 The RSA Inc. stream cipher. RC2 The RSA Inc. block cipher. IDEA An implmentation of the IDEA cipher, the library supports ecb, cbc, cfb64 and ofb64 modes of operation. Now all the above mentioned ciphers and digests libraries support high speed, minimal 'crap in the way' type interfaces. For fastest and lowest level access, these routines should be used directly. Now there was also the matter of public key crypto systems. These are based on large integer arithmatic. BN This is my large integer library. It supports all the normal arithmentic operations. It uses malloc extensivly and as such has no limits of the size of the numbers being manipulated. If you wish to use 4000 bit RSA moduli, these routines will handle it. This library also contains routines to 'generate' prime numbers and to test for primality. The RSA and DH libraries sit on top of this library. As of this point in time, I don't support SHA, but when I do add it, it will just sit on top of the routines contained in this library. RSA This implements the RSA public key algorithm. It also contains routines that will generate a new private/public key pair. All the RSA functions conform to the PKCS#1 standard. DH This is an implementation of the Diffie-Hellman protocol. There are all the require routines for the protocol, plus extra routines that can be used to generate a strong prime for use with a specified generator. While this last routine is not generally required by applications implementing DH, It is present for completeness and because I thing it is much better to be able to 'generate' your own 'magic' numbers as oposed to using numbers suplied by others. I conform to the PKCS#3 standard where required. You may have noticed the preceeding section mentions the 'generation' of prime numbers. Now this requries the use of 'random numbers'. RAND This psuedo-random number library is based on MD5 at it's core and a large internal state (2k bytes). Once you have entered enough seed data into this random number algorithm I don't feel you will ever need to worry about it generating predictable output. Due to the way I am writing a portable library, I have left the issue of how to get good initial random seed data upto the application but I do have support routines for saving and loading a persistant random number state for use between program runs. Now to make all these ciphers easier to use, a higher level interface was required. In this form, the same function would be used to encrypt 'by parts', via any one of the above mentioned ciphers. EVP The Digital EnVeloPe library is quite large. At it's core are function to perform encryption and decryption by parts while using an initial parameter to specify which of the 17 different ciphers or 4 different message digests to use. On top of these are implmented the digital signature functions, sign, verify, seal and open. Base64 encoding of binary data is also done in this library. PEM rfc???? describe the format for Privacy Enhanced eMail. As part of this standard, methods of encoding digital enveloped data is an ascii format are defined. As such, I use a form of these to encode enveloped data. While at this point in time full support for PEM has not been built into the library, a minimal subset of the secret key and Base64 encoding is present. These reoutines are mostly used to Ascii encode binary data with a 'type' associated with it and perhaps details of private key encryption used to encrypt the data. PKCS7 This is another Digital Envelope encoding standard which uses ASN.1 to encode the data. At this point in time, while there are some routines to encode and decode this binary format, full support is not present. As Mentioned, above, there are several different ways to encode data structures. ASN1 This library is more a set of primatives used to encode the packing and unpacking of data structures. It is used by the X509 certificate standard and by the PKCS standards which are used by this library. It also contains routines for duplicating and signing the structures asocisated with X509. X509 The X509 library contains routines for packing and unpacking, verifying and just about every thing else you would want to do with X509 certificates. PKCS7 PKCS-7 is a standard for encoding digital envelope data structures. At this point in time the routines will load and save DER forms of these structees. They need to be re-worked to support the BER form which is the normal way PKCS-7 is encoded. If the previous 2 sentances don't make much sense, don't worry, this library is not used by this version of SSLeay anyway. OBJ ASN.1 uses 'object identifiers' to identify objects. A set of functions were requred to translate from ASN.1 to an intenger, to a character string. This library provieds these translations Now I mentioned an X509 library. X509 specified a hieachy of certificates which needs to be traversed to authenticate particular certificates. METH This library is used to push 'methods' of retrieving certificates into the library. There are some supplied 'methods' with SSLeay but applications can add new methods if they so desire. This library has not been finished and is not being used in this version. Now all the above are required for use in the initial point of this project. SSL The SSL protocol. This is a full implmentation of SSL v 2. It support both server and client authentication. SSL v 3 support will be added when the SSL v 3 specification is released in it's final form. Now quite a few of the above mentioned libraries rely on a few 'complex' data structures. For each of these I have a library. Lhash This is a hash table library which is used extensivly. STACK An implemetation of a Stack data structure. BUF A simple character array structure that also support a function to check that the array is greater that a certain size, if it is not, it is realloced so that is it. TXT_DB A simple memory based text file data base. The application can specify unique indexes that will be enforced at update time. CONF Most of the programs written for this library require a configuration file. Instead of letting programs constantly re-implment this subsystem, the CONF library provides a consistant and flexable interface to not only configuration files but also environment variables. But what about when something goes wrong? The one advantage (and perhaps disadvantage) of all of these functions being in one library was the ability to implement a single error reporting system. ERR This library is used to report errors. The error system records library number, function number (in the library) and reason number. Multiple errors can be reported so that an 'error' trace is created. The errors can be printed in numeric or textual form. ==== ssluse.doc ======================================================== We have an SSL_CTX which contains global information for lots of SSL connections. The session-id cache and the certificate verificate cache. It also contains default values for use when certificates are used. SSL_CTX default cipher list session-id cache certificate cache default session-id timeout period New session-id callback Required session-id callback session-id stats Informational callback Callback that is set, overrides the SSLeay X509 certificate verification The default Certificate/Private Key pair Default read ahead mode. Default verify mode and verify callback. These are not used if the over ride callback mentioned above is used. Each SSL can have the following defined for it before a connection is made. Certificate Private key Ciphers to use Certificate verify mode and callback IO object to use in the comunication. Some 'read-ahead' mode information. A previous session-id to re-use. A connection is made by using SSL_connect or SSL_accept. When non-blocking IO is being used, there are functions that can be used to determin where and why the SSL_connect or SSL_accept did not complete. This information can be used to recall the functions when the 'error' condition has dissapeared. After the connection has been made, information can be retrived about the SSL session and the session-id values that have been decided upon. The 'peer' certificate can be retrieved. The session-id values include 'start time' 'timeout length' ==== stack.doc ======================================================== The stack data structure is used to store an ordered list of objects. It is basically misnamed to call it a stack but it can function that way and that is what I originally used it for. Due to the way element pointers are kept in a malloc()ed array, the most efficient way to use this structure is to add and delete elements from the end via sk_pop() and sk_push(). If you wish to do 'lookups' sk_find() is quite efficient since it will sort the stack (if required) and then do a binary search to lookup the requested item. This sorting occurs automatically so just sk_push() elements on the stack and don't worry about the order. Do remember that if you do a sk_find(), the order of the elements will change. You should never need to 'touch' this structure directly. typedef struct stack_st { unsigned int num; char **data; int sorted; unsigned int num_alloc; int (*comp)(); } STACK; 'num' holds the number of elements in the stack, 'data' is the array of elements. 'sorted' is 1 is the list has been sorted, 0 if not. num_alloc is the number of 'nodes' allocated in 'data'. When num becomes larger than num_alloc, data is realloced to a larger size. If 'comp' is set, it is a function that is used to compare 2 of the items in the stack. The function should return -1, 0 or 1, depending on the ordering. #define sk_num(sk) ((sk)->num) #define sk_value(sk,n) ((sk)->data[n]) These 2 macros should be used to access the number of elements in the 'stack' and to access a pointer to one of the values. STACK *sk_new(int (*c)()); This creates a new stack. If 'c', the comparison function, is not specified, the various functions that operate on a sorted 'stack' will not work (sk_find()). NULL is returned on failure. void sk_free(STACK *); This function free()'s a stack structure. The elements in the stack will not be freed so one should 'pop' and free all elements from the stack before calling this function or call sk_pop_free() instead. void sk_pop_free(STACK *st; void (*func)()); This function calls 'func' for each element on the stack, passing the element as the argument. sk_free() is then called to free the 'stack' structure. int sk_insert(STACK *sk,char *data,int where); This function inserts 'data' into stack 'sk' at location 'where'. If 'where' is larger that the number of elements in the stack, the element is put at the end. This function tends to be used by other 'stack' functions. Returns 0 on failure, otherwise the number of elements in the new stack. char *sk_delete(STACK *st,int loc); Remove the item a location 'loc' from the stack and returns it. Returns NULL if the 'loc' is out of range. char *sk_delete_ptr(STACK *st, char *p); If the data item pointed to by 'p' is in the stack, it is deleted from the stack and returned. NULL is returned if the element is not in the stack. int sk_find(STACK *st,char *data); Returns the location that contains a value that is equal to the 'data' item. If the comparison function was not set, this function does a linear search. This function actually qsort()s the stack if it is not in order and then uses bsearch() to do the initial search. If the search fails,, -1 is returned. For mutliple items with the same value, the index of the first in the array is returned. int sk_push(STACK *st,char *data); Append 'data' to the stack. 0 is returned if there is a failure (due to a malloc failure), else 1. This is sk_insert(st,data,sk_num(st)); int sk_unshift(STACK *st,char *data); Prepend 'data' to the front (location 0) of the stack. This is sk_insert(st,data,0); char *sk_shift(STACK *st); Return and delete from the stack the first element in the stack. This is sk_delete(st,0); char *sk_pop(STACK *st); Return and delete the last element on the stack. This is sk_delete(st,sk_num(sk)-1); void sk_zero(STACK *st); Removes all items from the stack. It does not 'free' pointers but is a quick way to clear a 'stack of references'. ==== threads.doc ======================================================== How to compile SSLeay for multi-threading. Well basically it is quite simple, set the compiler flags and build. I have only really done much testing under Solaris and Windows NT. If you library supports localtime_r() and gmtime_r() add, -DTHREADS to the makefile parameters. You can probably survive with out this define unless you are going to have multiple threads generating certificates at once. It will not affect the SSL side of things. The approach I have taken to doing locking is to make the application provide callbacks to perform locking and so that the SSLeay library can distinguish between threads (for the error state). To have a look at an example program, 'cd mt; vi mttest.c'. To build under solaris, sh solaris.sh, for Windows NT or Windows 95, win32.bat This will build mttest which will fire up 10 threads that talk SSL to each other 10 times. To enable everything to work, the application needs to call CRYPTO_set_id_callback(id_function); CRYPTO_set_locking_callback(locking_function); before any multithreading is started. id_function does not need to be defined under Windows NT or 95, the correct function will be called if it is not. Under unix, getpid() is call if the id_callback is not defined, for Solaris this is wrong (since threads id's are not pid's) but under Linux it is correct (threads are just processes sharing the data segement). The locking_callback is used to perform locking by the SSLeay library. eg. void solaris_locking_callback(mode,type,file,line) int mode; int type; char *file; int line; { if (mode & CRYPTO_LOCK) mutex_lock(&(lock_cs[type])); else mutex_unlock(&(lock_cs[type])); } Now in this case I have used mutexes instead of read/write locks, since they are faster and there are not many read locks in SSLeay, you may as well always use write locks. file and line are __FILE__ and __LINE__ from the compile and can be usefull when debugging. Now as you can see, 'type' can be one of a range of values, these values are defined in crypto/crypto.h CRYPTO_get_lock_name(type) will return a text version of what the lock is. There are CRYPTO_NUM_LOCKS locks required, so under solaris, the setup for multi-threading can be static mutex_t lock_cs[CRYPTO_NUM_LOCKS]; void thread_setup() { int i; for (i=0; i<CRYPTO_NUM_LOCKS; i++) mutex_init(&(lock_cs[i]),USYNC_THREAD,NULL); CRYPTO_set_id_callback((unsigned long (*)())solaris_thread_id); CRYPTO_set_locking_callback((void (*)())solaris_locking_callback); } As a final note, under Windows NT or Windows 95, you have to be careful not to mix the various threaded, unthreaded and debug libraries. Normally if they are mixed incorrectly, mttest will crash just after printing out some usage statistics at the end. This is because the different system libraries use different malloc routines and if data is malloc()ed inside crypt32.dll or ssl32.dll and then free()ed by a different library malloc, things get very confused. The default SSLeay DLL builds use /MD, so if you use this on your application, things will work as expected. If you use /MDd, you will probably have to rebuild SSLeay using this flag. I should modify util/mk1mf.pl so it does all this correctly, but this has not been done yet. One last warning. Because locking overheads are actually quite large, the statistics collected against the SSL_CTX for successfull connections etc are not locked when updated. This does make it possible for these values to be slightly lower than they should be, if you are running multithreaded on a multi-processor box, but this does not really matter much. ==== txt_db.doc ======================================================== TXT_DB, a simple text based in memory database. It holds rows of ascii data, for which the only special character is '\0'. The rows can be of an unlimited length. ==== why.doc ======================================================== This file is more of a note for other people who wish to understand why the build environment is the way it is :-). The include files 'depend' as follows. Each of crypto/*/*.c includes crypto/cryptlib.h ssl/*.c include ssl/ssl_locl.h apps/*.c include apps/apps.h crypto/cryptlib.h, ssl/ssl_locl.h and apps/apps.h all include e_os.h which contains OS/environment specific information. If you need to add something todo with a particular environment, add it to this file. It is worth remembering that quite a few libraries, like lhash, des, md, sha etc etc do not include crypto/cryptlib.h. This is because these libraries should be 'independantly compilable' and so I try to keep them this way. e_os.h is not so much a part of SSLeay, as the placing in one spot all the evil OS dependant muck. I wanted to automate as many things as possible. This includes error number generation. A make errors will scan the source files for error codes, append them to the correct header files, and generate the functions to print the text version of the error numbers. So don't even think about adding error numbers by hand, put them in the form XXXerr(XXXX_F_XXXX,YYYY_R_YYYY); on line and it will be automatically picked up my a make errors. In a similar vein, programs to be added into ssleay in the apps directory just need to have an entry added to E_EXE in makefile.ssl and everthing will work as expected. Don't edit progs.h by hand. make links re-generates the symbolic links that are used. The reason why I keep everything in its own directory, and don't put all the test programs and header files in 'test' and 'include' is because I want to keep the 'sub-libraries' independant. I still 'pull' out indervidual libraries for use in specific projects where the code is required. I have used the 'lhash' library in just about every software project I have worked on :-). make depend generates dependancies and make dclean removes them. You will notice that I use perl quite a bit when I could be using 'sed'. The reason I decided to do this was to just stick to one 'extra' program. For Windows NT, I have perl and no sed. The util/mk1mf.pl program can be used to generate a single makefile. I use this because makefiles under Microsoft are horrific. Each C compiler seems to have different linker formats, which have to be used because the retarted C compilers explode when you do cl -o file *.o. Now some would argue that I should just use the single makefile. I don't like it during develoment for 2 reasons. First, the actuall make command takes a long time. For my current setup, if I'm in crypto/bn and I type make, only the crypto/bn directory gets rebuilt, which is nice when you are modifying prototypes in bn.h which half the SSLeay depends on. The second is that to add a new souce file I just plonk it in at the required spot in the local makefile. This then alows me to keep things local, I don't need to modify a 'global' tables (the make for unix, the make for NT, the make for w31...). When I am ripping apart a library structure, it is nice to only have to worry about one directory :-). Having said all this, for the hell of it I put together 2 files that #include all the souce code (generated by doing a ls */*.o after a build). crypto.c takes only 30 seconds to build under NT and 2 minutes under linux for my pentium100. Much faster that the normal build :-). Again, the problem is that when using libraries, every program linked to libcrypto.a would suddenly get 330k of library when it may only need 1k. This technique does look like a nice way to do shared libraries though. Oh yes, as a final note, to 'build' a distribution, I just type make dist. This cleans and packages everything. The directory needs to be called SSLeay since the make does a 'cd ..' and renames and tars things up. ==== req.1 ======================================================== The 'req' command is used to manipulate and deal with pkcs#10 certificate requests. It's default mode of operation is to load a certificate and then write it out again. By default the 'req' is read from stdin in 'PEM' format. The -inform option can be used to specify 'pem' format or 'der' format. PEM format is the base64 encoding of the DER format. By default 'req' then writes the request back out. -outform can be used to indicate the desired output format, be it 'pem' or 'der'. To specify an input file, use the '-in' option and the '-out' option can be used to specify the output file. If you wish to perform a command and not output the certificate request afterwards, use the '-noout' option. When a certificate is loaded, it can be printed in a human readable ascii format via the '-text' option. To check that the signature on a certificate request is correct, use the '-verify' option to make sure that the private key contained in the certificate request corresponds to the signature. Besides the default mode, there is also the 'generate a certificate request' mode. There are several flags that trigger this mode. -new will generate a new RSA key (if required) and then prompts the user for details for the certificate request. -newkey has an argument that is the number of bits to make the new key. This function also triggers '-new'. The '-new' option can have a key to use specified instead of having to load one, '-key' is used to specify the file containg the key. -keyform can be used to specify the format of the key. Only 'pem' and 'der' formats are supported, later, 'netscape' format may be added. Finally there is the '-x509' options which makes req output a self signed x509 certificate instead of a certificate request. Now as you may have noticed, there are lots of default options that cannot be specified via the command line. They are held in a 'template' or 'configuration file'. The -config option specifies which configuration file to use. See conf.doc for details on the syntax of this file. The req command uses the 'req' section of the config file. --- # The following variables are defined. For this example I will populate # the various values [ req ] default_bits = 512 # default number of bits to use. default_keyfile = testkey.pem # Where to write the generated keyfile # if not specified. distinguished_name= req_dn # The section that contains the # information about which 'object' we # want to put in the DN. attributes = req_attr # The objects we want for the # attributes field. encrypt_rsa_key = no # Should we encrypt newly generated # keys. I strongly recommend 'yes'. # The distinguished name section. For the following entries, the # object names must exist in the SSLeay header file objects.h. If they # do not, they will be silently ignored. The entries have the following # format. # <object_name> => string to prompt with # <object_name>_default => default value for people # <object_name>_value => Automatically use this value for this field. # <object_name>_min => minimum number of characters for data (def. 0) # <object_name>_max => maximum number of characters for data (def. inf.) # All of these entries are optional except for the first one. [ req_dn ] countryName = Country Name (2 letter code) countryName_default = AU stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Queensland localityName = Locality Name (eg, city) organizationName = Organization Name (eg, company) organizationName_default = Mincom Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) organizationalUnitName_default = MTR commonName = Common Name (eg, YOUR name) commonName_max = 64 emailAddress = Email Address emailAddress_max = 40 # The next section is the attributes section. This is exactly the # same as for the previous section except that the resulting objects are # put in the attributes field. [ req_attr ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name ---- Also note that the order that attributes appear in this file is the order they will be put into the distinguished name. Once this request has been generated, it can be sent to a CA for certifying. ---- A few quick examples.... To generate a new request and a new key req -new To generate a new request and a 1058 bit key req -newkey 1058 To generate a new request using a pre-existing key req -new -key key.pem To generate a self signed x509 certificate from a certificate request using a supplied key, and we want to see the text form of the output certificate (which we will put in the file selfSign.pem req -x509 -in req.pem -key key.pem -text -out selfSign.pem Verify that the signature is correct on a certificate request. req -verify -in req.pem Verify that the signature was made using a specified public key. req -verify -in req.pem -key key.pem Print the contents of a certificate request req -text -in req.pem ==== danger ======================================================== If you specify a SSLv2 cipher, and the mode is SSLv23 and the server can talk SSLv3, it will claim there is no cipher since you should be using SSLv3. When tracing debug stuff, remember BIO_s_socket() is different to BIO_s_connect(). BSD/OS assember is not working
Back to File Manager