Introducing Top Secret

We’ve written about how to prevent logging sensitive information when making network requests, but that approach only works if you’re dealing with parameters.

What happens when you’re dealing with free text? Filtering the entire string may not be an option if an external API needs to process the value. Think chatbots or LLMs.

You could use a regex to filter sensitive information (such as credit card numbers or emails), but that won’t capture everything, since not all sensitive information can be captured with a regex.

Fortunately, named-entity recognition (NER) can be used to identify and classify real-world objects, such as a person, or location. Tools like MITIE Ruby make interfacing with NER models trivial.

By using a combination of regex patterns and NER entities, Top Secret effectively filters sensitive information from free text—here are some real-world examples.

If you want to see Top Secret in action, you might enjoy this live stream. Otherwise, see the examples below.

Working with LLMs

It’s not uncommon to send user data to chatbots. Since the data might be free-form, we should be diligent about filtering it using the approach mentioned above.

However, it’s likely we’ll want to “restore” the filtered values when returning a response from the chatbot. Top Secret returns a mapping that would allow for this.

You’d likely want to provide instructions in the request.

instructions = <<~TEXT
  I'm going to send filtered information to you in the form of free text.
  If you need to refer to the filtered information in a response, just reference it by the filter.
TEXT

The exchange might look something like this.

1. Caller sends filtered text

   result = TopSecret::Text.filter("Ralph lives in Boston.")
   
   # Send this to the API
   result.output # => [PERSON_1] lives in [LOCATION_1].
   
   # Save the mapping to "restore" response
   mapping = result.mapping # => { PERSON_1: "Ralph", LOCATION_1: "Boston" }
   

2. API responds with filter

   "Hi [PERSON_1]! How is the weather in [LOCATION_1] today?"
   

3. Caller can “restore” from the mapping

   response = "Hi [PERSON_1]! How is the weather in [LOCATION_1] today?"
   
   # Restore the response from the mapping
   result = TopSecret::FilteredText.restore(response, mapping: mapping)
   
   result.output
   # => Hi Ralph! How is the weather in Boston today?
   

Filtering conversation history

When working with conversation state you should filter every message before including it in the request. This ensures no sensitive data slips through from previous messages. Here’s what that might look like.

require "openai"
require "top_secret"

openai = OpenAI::Client.new(
  api_key: Rails.application.credentials.openai.api_key!
)

original_messages = [
  "Ralph lives in Boston.",
  "You can reach them at ralph@thoughtbot.com or 877-976-2687"
]

# Filter all messages
result = TopSecret::Text.filter_all(original_messages)
filtered_messages = result.items.map(&:output)

user_messages = filtered_messages.map { {role: "user", content: it} }

# Instruct LLM how to handle filtered messages
instructions = <<~TEXT
  I'm going to send filtered information to you in the form of free text.
  If you need to refer to the filtered information in a response, just reference it by the filter.
TEXT

messages = [
  {role: "system", content: instructions},
  *user_messages
]

chat_completion = openai.chat.completions.create(messages:, model: :"gpt-5")
response = chat_completion.choices.last.message.content

# Restore the response from the mapping
mapping = result.mapping
restored_response = TopSecret::FilteredText.restore(response, mapping:).output

puts(restored_response)

Prevent storing sensitive information with validations

Top Secret can also be used as a validation tool to prevent storing sensitive information in your database.

class Message < ApplicationRecord
  validate :content_cannot_contain_sensitive_information

  private

  def content_cannot_contain_sensitive_information
    result = TopSecret::Text.filter(content)
    return if result.mapping.empty?

    errors.add(:content, "contains the following sensitive information #{result.mapping.values.to_sentence}")
  end
end

If the validation is too strict, you can override or disable any of the filters as needed.

--- a/app/models/message.rb
+++ b/app/models/message.rb
@@ -4,7 +4,7 @@ class Message < ApplicationRecord
   private

   def content_cannot_contain_sensitive_information
-    result = TopSecret::Text.filter(content)
+    result = TopSecret::Text.filter(content, people_filter: nil, location_filter: nil)
     return if result.mapping.empty?

     errors.add(:content, "contains the following sensitive information #{result.mapping.values.to_sentence}")

Wrapping up

It’s our responsibility to protect user data. This is more important than ever given the rise in popularity of chatbots and LLMs. Tools like Top Secret aim to reduce this burden.