A close up of a typewriter

German World War II Enigma encryption machine

Public Key Encryption: A Beginner's Guide

DISCLAIMER: 100% of the content in this blog post is original and has been created by humans, including its research, writing, images, and graphics.  No AI (Artificial Intelligence) was used to create any portion of the content.

TL;DR: Encryption is the process by which sensitive information is intentionally scrambled so that it becomes unintelligible.  Once it becomes unintelligible, sensitive information is protected from abuse by unauthorized persons because it can no longer be read, understood, or modified.  This article explains how the secret key encryption that was used as far back in history as the Roman Empire ("Caesar's Cipher") works and why it was the only means of encryption available until 1976 when the Diffie-Hellman method of public key encryption was invented.  The operation of Diffie-Hellman (which is explained in detail) led to the invention of RSA public key/private key encryption in 1977.  RSA is the predominant method of encryption is use today; its entire "cryptosystem" is explained in detail in this article.  You will learn how to determine whether you can use "passive" encryption to meet your information security needs or if a dedicated encryption application is required.  If you are interested in testing the security of your sensitive information, please visit our home page.

Learning what public key encryption is and how it works will enable you to understand why it's the predominant method of protecting sensitive information.  This blog post is written specifically for those who are new to public key encryption and who want to gain a functional "beginner level" understanding of this critically important technology.

This post is a comprehensive presentation that defines the terms associated with public key encryption and the context in which those terms are used.  Those readers who already have a basic understanding of public key encryption may find this post long and tedious.  But for those of you without any understanding of public key encryption, the time you spend reading this post will reward you with the knowledge to be able to "hold your own" in all but the most technical conversations about public key encryption.  Even those of you who have a basic understanding of public key encryption may find the latter sections of this post useful — "The Challenges of Using Encryption Applications," and "Do I Need An Encryption Application?"

Encrypting information is a complex topic.  Public key encryption, today's most widely used system of encrypting sensitive data, is one of the most complicated technologies ever invented.  People typically turn to the Internet when they first begin to learn about public key encryption and quickly discover that there is an an enormous volume of information about all types of encryption, including public key encryption.  Unfortunately, most of this information is confusing because it is written by technical people, for technical people.  It is full of jargon that newcomers to encryption have no background in that would enable them to understand what they are reading.  Most people give up trying to learn about encryption because they never find an explanation of encryption that they can understand.

This post will provide you with the necessary background about encryption so that you will be able to understand how public key encryption functions and why it is the encryption of choice that is used to protect sensitive information.

Definitions

Encryption is the process by which some form of sensitive content is intentionally scrambled so that the content becomes unintelligible.  By becoming unintelligible, the sensitive content is protected because it can no longer be understood.  Almost every form of sensitive content can be encrypted.  Examples of the most common forms of sensitive content that are routinely encrypted are:

Various methods are used to scramble (encrypt) sensitive content, depending upon the nature of the content.  The field of study that invents the methods that are used to encrypt sensitive content is known as cryptography.  Every encryption process uses one or more cryptography methods to encrypt sensitive data.  Unfortunately, due to the lack of understanding on the part of the general public about their differences, the terms "encryption" and "cryptography" are used synonymously.  Because of this, the terms "public key encryption" and "public key cryptography" are both routinely used to refer to the process of encrypting sensitive content — even though the technically correct term is "public key encryption."  This is a significant factor that contributes to the difficulty of learning about encryption, especially on the Internet.

The remainder of this blog post will focus on explaining what public key encryption is and how it works — at a beginner level.  References will be made to the use of cryptography to encrypt sensitive content, but an explanation of cryptography, per se, is beyond the scope of this blog post.

In The Beginning …

Like almost all technologies, public key encryption evolved from a simpler version of encryption — secret key encryption.  Secret key encryption has been used for centuries and is still in use today.  Someone who wants to encrypt sensitive content using secret key encryption either creates a new, or uses an exiting, cryptographic method to encrypt the content.  The cryptographic method used to encrypt the sensitive content is known as a "cryptographic key" (shortened to just "key" in common use) because the method is used to metaphorically "lock" and "unlock" the sensitive content.  The person who wishes to encrypt sensitive content uses the key to encrypt (scramble or "lock") the content.  Only those persons who possess the key can "unlock" the sensitive content to use it.  Secret key encryption requires that both the person who encrypts the sensitive content and the person who "unlocks" (decrypts) the sensitive content must possess the secret key.  The cartoon below depicts how Caesar, the Roman emperor, used secret key encryption.

A cartoon of a person in a robe Description automatically generated

While we are using a cartoon to illustrate how secret key encryption works, Caesar actually did use a 3-left cipher to encrypt sensitive content. It's known today as "Caesar's Cipher."  We'll explain how Caesar used his cipher to encrypt his message in just a minute.  First, however, there are new terms that need to be defined so that the cartoon makes sense:

In the cartoon above, the cipher used to encrypt Caesar's message to Tiberius Nero is shown directly above the two messages.  The "Plain" alphabet is overlaid on top of the "Cipher" alphabet.  To find the encrypted character (the Cipher character) for any original character (the Plain character) the person encrypting Caesar's message would first find the Plain character then look below the Plain character to find the Cipher character.  You can easily confirm that Plain characters are being replaced with Cipher characters that are three characters to the left of the Plain character by examining the cipher closely.  For example, the Plain character "E" is replaced by the character in the alphabet three positions to its left, which is the character "B."

While not shown as a term in the cartoon above, remember that encryption algorithms specify the encryption process by which a cipher is used to encrypt sensitive content, while the cipher itself specifies the cryptographic method used to actually encrypt the content.  The encryption algorithm for encrypting Caesar's message uses the cipher to replace the Plain characters with Cipher characters in a copy of the original message.  

By way of making you aware, the term "secret key," which is a cipher and not an encryption algorithm, is often used in a sentence such as, "The secret key was used to encrypt the …," which implies that a secret key is both a cipher and an encryption algorithm.  This is not true — a secret key is not an encryption algorithm — it is a cipher only.  Similarly, the sentence, "The secret key algorithm was used to encrypt …" is often used to describe the act of encrypting sensitive information, even though a secret key is not an encryption algorithm.  You will find that these usages of the term "secret key" are so prevalent that we have elected to use both of them in this post to mean the same thing as, "The secret key and the encryption algorithm were used together to encrypt the …"  

Once Caesar's message has been encrypted, the ciphertext (the encrypted message) can be delivered to Tiberius Nero without fear of the message being read by unauthorized persons.  Only those persons who posses the secret key can decrypt (decode) the ciphertext message.  Assuming that Caesar has provided his secret key to only Tiberius Nero, then only Tiberius Nero will be able to decrypt the ciphertext message.  When Tiberius Nero receives the ciphertext message he is able to decrypt is as shown below.

A cartoon of a person in a garment Description automatically generated

After receiving the ciphertext message, Tiberius Nero decrypts it using the same cipher that Caesar used to encrypt the message — only in reverse order.  Nero compares each character from the ciphertext message against the cipher and replaces the ciphertext character with its corresponding plaintext character in a copy of the ciphertext message.  As long as the secret key is known only to Caesar and Tiberius Nero, any encrypted messages exchanged between them using the secret key will remain secret.

Modern Day Encryption

The basic concepts underpinning modern encryption haven't changed since Caesar's time — only the technologies used to perform encryption have changed.  Sensitive information, shared between two or more parties, needs to be encrypted so that only those parties who possess the secret key can use the encrypted information.  To everyone else the encrypted information is unintelligible.  But even in Caesar's time there were significant shortcomings associated with the use of secret key encryption:

Even though the issues listed above existed in Caesar's time, they were manageable because the need to encrypt sensitive data was limited to government use.  With the exception of of a small number of nobles, the general pubic in Caesar's time had no awareness of, or need for, encryption.  Fast forward to today.  Without encryption, the modern world couldn't exist.  Any activity conducted over the public Internet (online) that requires privacy uses encryption to provide that privacy.  Email, banking, securities trading, web browsing, messaging, military communications, government transactions, etc., etc., etc., couldn't function without encryption to provide privacy.  There are methods other than encryption that could be used to keep sensitive information private (e.g., steganography) but none of them have the reliability, flexibility, and low cost offered by encryption.  Because of this, encryption has become today's global standard for protecting sensitive information.

Public Key and Private Key Encryption

In 1976 three computer scientists, Whitfield Diffie, Martin Hellman, and Ralph Merkle devised a method of securely generating a shared secret key (which later became known as a private key) using an insecure medium (a public computer network) without the need to send the shared secret key over the computer network.  This encryption method became known as the Diffie-Hellman key exchange algorithm (shortened in common usage to "Diffie-Hellman") and was the first use of a public key to generate a shared secret key.  In Diffie-Hellman, each party who wants to exchange encrypted messages uses the information from a public key that is available to all parties to generate on their individual computers the shared secret key that is used to encrypt and decrypt messages.  Because the shared secret key is generated on each party's computer, it is never sent over the computer network and can never be captured by network eavesdroppers.  The public key, as its name implies, can be seen by anyone in the public who has access to the computer network, including network eavesdroppers.  But the shared private key, never being sent over the network, can never be seen by anyone but the parties to the encrypted message exchange.  NOTE: This description of the operation of Diffie-Hellman is very high level and doesn't attempt to convey the details of Diffie-Hellman because this post is a beginner's guide.  An illustration of the operation of Diffie-Hellman is shown below.

A computer screen with a key and arrows Description automatically generated

Diffie-Hellman solved the longstanding shortcoming of secret key confidentiality that had existed until 1976.  It was revolutionary because it limited the distribution of secret keys to only those parties who were active participants in an encrypted message exchange and it generated new secret keys for each new encrypted message exchange, which meant that existing secret keys couldn't be used to decrypt new message exchanges.

RSA Cryptosystem

In 1977, one year following the release of Diffie-Hellman, a new public key and private key cryptography system was released by Ron Rivest, Adi Shamir, and Leonard Adleman who worked at the Massachusetts Institute of Technology (MIT).  Named "RSA" (using the first letter of each inventor's last name), this public key and private key invention was both a shared secret key cipher generator (using a public key) and an encryption algorithm.  Because RSA contains both the cipher generator (the cryptography method) and the encryption algorithm (the encryption process), RSA could both create the shared secret key (hereinafter referred to as the "private key") then use the private key to encrypt sensitive information.  This was a huge improvement over Diffie-Hellman since Diffie-Hellman could only create the private key and didn't actually perform the operation of encrypting sensitive information using the private key.

RSA included features that catapulted it to become the most used public key encryption in the world.  First, it enabled anyone to create a private key, from which a mathematically linked public key was created.  Messages encrypted using the public key could be decrypted using the private key.  Conversely, messages created with the private key could be decrypted using the public key.  RSA not only enables the creation of public and private keys, it uses the keys to actually encrypt and decrypt sensitive information.  RSA also introduced a new feature known as "message signing."  Message signing enables the creator of an encrypted message to "sign" their encrypted message using their private key.  When the "signed" message is decoded by the receiving party, the receiving party is able to confirm two important facts about the encrypted message:

The introduction of RSA's message signing capability solved another one of the four shortcomings of encryption — Information Integrity.  Additionally, while not providing 100% Identity Verification (and solving this encryption shortcoming), RSA message signing does provide verification of the private key used to sign a message, which if the private key has not been compromised (e.g., lost, stolen, copied, etc.), functions as Identity Verification.  It should now be clear why RSA is commonly referred to as the "RSA cryptosystem"  — because it provides much more functionality that just encryption.  A high-level, conceptual illustration of the operation of RSA is shown below.  The actual behavior of RSA as it is implemented in various encryption software products is usually slightly different that this conceptual illustration.

A computer screen shot of a cartoon Description automatically generated with medium confidence

Understanding the RSA Message Signature

How the RSA message signature is created, then used to verify that the received message has not been tampered with, can be confusing.  The process starts in Step #3 of the illustration, above.  The plaintext message is processed by a hash function that generates a single, unique number that can only be produced by using the plaintext message as the input to the hash function.  A hash function is a small piece of software code that is part of an encryption application's overall code.  It's purpose is to create a number that uniquely identifies a single piece of digital content, similar to how serial numbers are used to identify individual products.  Any digital content can be used as the input to a hash function — text, images, files, music, video, etc.  The hash function is what's known as a "one-way function," which means that the number generated by the hash function for a piece of digital content can't be used to recreate the digital content.  This makes the hash function 100% secure and an ideal tool to identify digital content because a hash number can never be used to divulge the digital content from which it was created.

In Step #3 of the illustration, above, the hash number of the plaintext message is generated as a plaintext number in hexadecimal format (A7:6D:08:E2:9F:B6:05:03:F5:CF).  The plaintext hash number is then encrypted (UubjhRQ789!^|9Sdt!+0uR) using the private key of the message sender (in Step #3 this is Alice's private key) so that the plaintext hash number can't be tampered with.  After the plaintext hash number has been generated and encrypted, the plaintext message is encrypted using the recipient's public key (in Step #3 this is Bob's public key) so that it can't be tampered with.  The encrypted hash number is added to the encrypted plaintext message as the message signature (which is known as "signing the message").  Alice then sends the encrypted and signed message to Bob.

When the encrypted and signed message is received by Bob in Step #5 of the illustration, above, Bob's encryption software gets to work decrypting the ciphertext message and confirming that the decrypted plaintext message hasn't been tampered with.  The decryption activities performed in Step #5 are described below.

The use of the message signature to validate that the ciphertext message has not been tampered with, as described above, is the method described by the RSA specification.  In practice, there are many variations on how the RSA encryption and signing algorithm is implemented in encryption software applications.  Some of the challenges these application implementations pose to the use of encrypted messages between message exchange participants is presented in the next section of this guide.

The Challenges of Using Encryption Applications

Neither Diffie-Hellman nor RSA is a software application that you can buy — they are encryption specifications (two of many) that are used by software companies to develop their encryption software applications.  Because there are many encryption specifications, the functionality of encryption applications varies widely, which means that using encryption applications can be more challenging for some people than for others.  The major challenges you will encounter if you choose to use an encryption application are:

  1. Encryption Is Not Available For Your Application — You may be using a software application that doesn't include native encryption functionality.  You search for third-party encryption applications that can be added to your application (e.g., a plug-in or add-in) but nothing can be found.  In this situation you won't be able to use any encryption with your application.
  2. Encryption Is Not Compatible On All Operating Systems And Devices — Your application can use encryption (whether it's natively supplied or is available via a third-party encryption product like an add-in, plug-in, update, etc.) but it either doesn't work at all or has limited encryption functionality when used on other operating systems.  The biggest issue in this regard is not having encryption work seamlessly between the Windows and MacOS operating systems.  Trying to get encryption to work seamlessly between all major device operating systems (i.e., Windows, MacOS, Android, and iOS) to support sending messages between desktop computers, laptop computers, tablets, and smartphones probably won't happen unless a single third-party encryption application provides seamless support across all operating systems (because these operating systems don't provide seamless encryption between themselves).
  3. Each Person You Want To Exchange Messages With Has To Be Contacted — Because each encryption software application provides specific encryption functionality, every one of the people who you wish to exchange encrypted messages with must be contacted in advance to see if their encryption software can work with your encryption software.  These contacts may require the involvement of IT departments to determine whether or not the involved encryption applications can work together.  For any organization this can be a time-consuming process, but for mid-size and large organizations this becomes a burden so large that a dedicated team of IT encryption specialist may need to be hired.
  4. The Exchange Of Public Keys Is Typically A Manual Process — Even after you (or your IT department) have confirmed that your encryption will work with your message recipients' encryption, your public key must be shared with each recipient and they must each share their public key with you.  This is typically done by either sending an unencrypted email that you have signed with your private key to each recipient (which contains your public key), using an S/MIME certificate that contains your public key, or arranging for your recipients to somehow securely receive your public key as a digital file (on USB flash drive, for example).  However you choose to handle the exchange of public keys, this is a time-consuming process in addition to the time spent verifying that encryption works for all parties to your message exchange.
  5. Reliance On A Single Commercial or Open Source Encryption Vendor — Whatever encryption application you choose to use, you will be reliant upon a single commercial or open source vendor to support your use of their encryption application over time.  If the vendor fails to keep pace with changes in encryption technology over time, or goes out of business, your ability to reliably use their encryption application will fade with time.  Selecting a reputable commercial or open source encryption vendor who has a demonstrated track record of staying at the forefront of encryption technology and who has a large encryption application market share is the best way to mitigate this challenge.
  6. Application Maintenance; New Encryption Technologies — Mentioned in Challenge #5, above, the quality of application maintenance and upkeep is critically necessary for encryption applications so that your organization can exchange encrypted messages knowing that they can't be decrypted by unauthorized persons.  The only way this can happen is if encryption application vendors stay current with the latest developments in encryption technology and regularly incorporate those developments into their application.  This will be especially import as quantum computing becomes commercially available and threatens to defeat all exiting encryption technologies.  Application maintenance and upkeep certainly involves the application vendor, but it also involves resources from your organization to take care of those tasks that are not the responsibility of the application vendor.
  7. Recovering From Security Breaches — If the public or private keys of any one of the participants in your encrypted message exchange is compromised (i.e., lost, stolen, copied, tampered with, etc.) your entire message exchange is at risk of being compromised.  In this case, the public and private keys of the person(s) compromised will have to be re-generated and their new public key will have to be distributed to all message exchange participants.  Additionally, all messages sent by the person(s) whose key(s) have been compromise will need to be reviewed to make sure that confidential, misinformation, or disinformation hasn't been sent to message exchange participants.

It should now be apparent that encryption applications pose a number of challenges that require significant resources to ensure that they can be reliably used as desired.  Importantly, these resources must be available to support the application over time so that the application's reliability is never compromised.  Encryption is a technology that can't operate correctly 99% of the time.  It has to operate correctly 100% of the time or there is no guarantee that every encrypted message is protected from being read by unauthorized persons.

Do I Need An Encryption Application?

In order to answer this question, you first need to understand where data resides in your computer and which encryption mechanisms can be used to encrypt the data.  Data resides in only one of three "states" in your computer at any given point in time:

Now that you understand where data resides, answering the question, "Do I Need An Encryption Application," is simply a matter of quantifying the compromise risk associated with each data state and deciding whether the risk is so great that you need to encrypt the data to protect it while it's in that state.

At first glance, the compromise risk of some data states appears to be lower or higher than other data states.  For example, the risk of compromise for At Rest data seems like it would be higher than the risk for In Use data, because it seems like compromising data stored on a hard drive would be easier for hackers than compromising data that is in memory.  While the technical skills required to compromise data in memory are more demanding than the technical skills required to compromise data on a hard drive, the truth of the matter is that once a hacker has gained control of your computer they can access data in any of its states.  This means that the decision about using an encryption application should be made around the question, "If data is compromised in any of its states, do we need an encryption application to protect the data?"

The answer to the above question will depend upon the nature of the data and the risk that data compromise poses to the entity to which the data belongs.  If the CIA is asking the question, the answer would be a resounding, "Yes."  But what about the majority of entities whose data doesn't rise to the level of national secrets or intellectual property?  For these entities, in most cases, the answer would be, "No," because they are confident that the risk of compromise for their At Rest and In Use data is within their control.  But what about In Motion data?  That's a little trickier because In Motion data can be in the process of being transferred over a private network (which is in the control of the entity) or over the public Internet, which is completely out of control of the entity.

If only In Motion data being transferred over the public Internet is required to be encrypted, is an encryption application required?  Fortunately, the answer to this question is, "No."  A cryptographic protocol named Transport Layer Security ("TLS"), formerly know as Secure Sockets Layer ("SSL"), is available that encrypts In Motion data before it enters the public Internet.  The beauty of TLS is that it works seamlessly on all major operating systems and devices, it doesn't require any sharing of keys or contacting message exchange participants, and there are free open source as well as commercial TLS products.  TLS, like encryption applications, will require IT to set it up and maintain it, but end users never interact with TLS, unlike encryption applications that require the ongoing involvement of end users.

Conclusion

Believe it or not, this is a beginner's guide to public key encryption!  There is so much more technical detail surrounding public key encryption that has not been presented in this post.  As an introduction to this complex subject, we believe that this post will serve as a valuable resource for those persons who desire a working understanding of how encryption works and how it is used to protect sensitive information.  Without encryption, today's 21st century society couldn't exist.  Whether protecting government secrets, banking information, healthcare information, or the myriads of other sensitive information, encryption is one of the most significant underpinnings of today's global economy.

At Information Security Testing we create and support computerized testing products and services that reliably and accurately determine if digital information is secure.  We invite you to learn about our testing products and how you can determine if your sensitive information is secure by visiting our home page.