Nos últimos dois meses, várias pessoas me procuraram para saber como organizo meus estudos. Decidi, então, compilar um resumo das minhas técnicas e métodos. Espero que encontrem aqui ferramentas úteis!

Método de estudos

Primeiramente, o método de estudo é algo muito pessoal. As práticas que mencionarei tiveram êxito para mim. A primeira e mais importante tarefa é começar a desenvolver o seu método:

• Com que frequência você deseja estudar?
• Como você vai organizar o conhecimento?
• Que tipo de conteúdo mais te engaja (livros, vídeo aulas, cursos, etc)?
• Quanta estrutura você quer dar aos seus estudos? Você funciona melhor com um planejamento detalhado e prazos, ou prefere algo mais flexível, seguindo os tópicos que mais capturam sua atenção no momento?

Posso falar por mim: estudo todos os dias, leio artigos ou livros sempre que possível. Organizo meu conhecimento em notas no Obsidian, usando uma estrutura que combina o Zettelkasten com o método PARA. Prefiro, de longe, conteúdos escritos e não gosto de fazer grandes planejamentos; estudo o que tenho vontade, quando tenho vontade.

Fontes de informação

Explico que existem duas formas de desenvolvimento: horizontal e vertical.

• O desenvolvimento horizontal foca em criar uma base de conhecimento em um universo, abrangendo vários tópicos. Isso não implica em um estudo superficial, mas em entender como os tópicos se relacionam em um contexto. Dependendo do universo, como em segurança da informação, esse tipo de estudo pode ser um dos mais desafiadores.
• O desenvolvimento vertical, por sua vez, foca em aprofundar um tópico e aprender o máximo possível sobre ele. O objetivo é estudar como se quisesse se tornar um especialista naquele assunto.

Na carreira, é essencial manter ambos os tipos de desenvolvimento ativos. Para isso, recomendo realizar uma curadoria constante de diferentes tipos de fontes de informação:

• Newsletters (horizontal): para mim, são a melhor ferramenta para um profissional de tecnologia hoje. Faço uma seleção contínua das newsletters que assino. Ao receber uma nova publicação, seleciono links relevantes, adiciono-os à minha lista de leitura e os consumo gradualmente.
• Feeds de agregadores (horizontal): sites como Hackersnews e Lobste.rs têm feeds RSS organizados por tema. Funcionam como uma linha do tempo de uma rede social, com muito ruído, mas ocasionalmente você encontra artigos úteis. É crucial saber gerenciar fontes de alto e baixo ruído para que não interfiram na eficiência do seu consumo.
• Blogs/Publicações especializadas (vertical): prefiro separar essas fontes dos livros tradicionais porque geralmente são mais sucintas, têm mais exemplos e exercícios práticos e são mais atualizadas que os livros. Exemplos incluem: https://quii.gitbook.io/learn-go-with-tests e https://www.fuzzingbook.org/.
• Papers (vertical): começar a ler artigos acadêmicos pode ser desafiador, mas é um passo importante para conhecer a vanguarda do desenvolvimento. Quanto antes começar, melhor.
• Livros e cursos (vertical): geralmente os utilizo para tópicos que evoluem mais lentamente, como algoritmos, matemática e criptografia. Nessas áreas, você encontrará publicações excelentes e ainda relevantes.

Com o tempo, ao consumir conteúdos, você acumulará links, artigos, livros, ferramentas: todo tipo de referência. Cuide bem dessa coleção, escolha uma forma de organização, um aplicativo, o que preferir. Essa coleção se tornará seu jardim, de onde extrairá insumos para produzir conhecimento.

No entanto, é fundamental não cair na falácia do colecionador, onde simplesmente acumulamos pilhas de materiais e nos damos por satisfeitos.

Organizando conhecimento

Um dos meus maiores arrependimentos foi não ter começado a manter notas organizadas mais cedo! Minha maior recomendação aqui é a simplicidade. É muito fácil cair na armadilha de otimizar a estrutura das suas notas e ser extremamente produtivo (eu já caí várias vezes), criando um sistema demasiadamente complicado.

Algumas coisas que faço para estruturar o conhecimento são:

• Notas de Literatura: fazer notas detalhadas e com suas próprias palavras custa muito tempo, então eu criei um tipo diferente de nota chamada Nota de Literatura. Nela, deposito informações relevantes da fonte de informação, por vezes copiando diretamente, outras vezes reescrevendo.
  • Assistente AI: embora o objetivo seja produzir notas mais simples e rápidas, a AI tem me ajudado a produzir notas de literatura com qualidade de notas de assunto. Eu uso um prompt que gera uma lista de perguntas sobre os tópicos abordados na fonte de informação. Isso tem tornado minhas notas de literatura muito mais eficientes. O prompt que uso está disponível publicamente: thought_provoking prompt.
• Notas de Assunto (Subject/Evergreen notes): essas notas são o personagem principal. Elas têm uma vida longa, são atualizadas sempre que aprendo algo novo sobre o assunto e são escritas com minhas próprias palavras. A técnica mais importante para mim aqui é escrever como se estivesse explicando para outra pessoa.

Aplicando seu método

Existem ainda desafios únicos durante o andamento dos seus estudos. Uma pergunta recorrente é: eu sei qual assunto quero estudar, sei qual tipo de estudo (horizontal ou vertical) e tenho uma curadoria de fontes - como escolher qual material consumir? Nesse caso, meu método é:

1. Seleciono material relacionado ao tema que já possa ter armazenado;
2. Uso DORKs para pesquisar material no histórico de newsletters e feeds de agregadores.
3. Dedico um tempo navegando entre os diferentes materiais selecionados, até encontrar um (ou alguns) com um bom equilíbrio entre didática e cobertura do assunto.

Isso acaba levando a um começo mais lento, porém, na minha experiência, ele se paga, pois me dá a oportunidade de selecionar materiais de estudo de maior qualidade.

Outro desafio comum é: como tirar dúvidas sobre o assunto? Atualmente, LLMs têm sido de grande ajuda para mim (desde que você saiba como lidar com possíveis alucinações). Buscar por comunidades dedicadas ao assunto no Reddit também costuma ter bons resultados. No entanto, a melhor ferramenta é sua própria coleção de materiais: busque por outros materiais que possam explicar o mesmo conceito a partir de diferentes pontos de vista. Eventualmente, você irá montar um quebra-cabeça, com uma compreensão muito mais abrangente do assunto.

Conclusão

A jornada do aprendizado é profundamente pessoal e varia de indivíduo para indivíduo. As estratégias e métodos que compartilhei refletem minhas experiências e preferências pessoais. É importante encontrar o equilíbrio entre a absorção de informação e a criação de conhecimento, evitando a armadilha de apenas acumular referências.

Lembre-se de que a chave para um estudo eficaz não está apenas em selecionar os materiais certos, mas também em como você organiza e interage com essas informações ao longo do tempo. Espero que este guia sirva como um ponto de partida para você desenvolver um método de estudo que seja tão único e eficaz quanto suas próprias aspirações e necessidades de aprendizado.

Over the past two months, several people have approached me to find out how I organize my studies. Therefore, I decided to compile a summary of my techniques and methods. I hope you find useful tools here!

Study Method

Firstly, the study method is something very personal. The practices I will mention have been successful for me. The first and most important task is to start developing your method:

• How often do you want to study?
• How will you organize knowledge?
• What type of content engages you the most (books, video lessons, courses, etc)?
• How much structure do you want to give your studies? Do you function better with detailed planning and deadlines, or do you prefer something more flexible, following the topics that capture your attention at the moment?

I can speak for myself: I study every day, read articles or books whenever possible. I organize my knowledge in notes in Obsidian, using a structure that combines the Zettelkasten method with the PARA method. I vastly prefer written content and do not like making extensive plans; I study what I want, when I want.

Sources of Information

I explain that there are two forms of development: horizontal and vertical.

• Horizontal development focuses on creating a knowledge base in a universe, covering various topics. This does not imply superficial study but understanding how the topics relate in a context. Depending on the universe, such as in information security, this type of study can be one of the most challenging.
• Vertical development, on the other hand, focuses on deepening a topic and learning as much as possible about it. The goal is to study as if you wanted to become an expert on that subject.

In your career, it is essential to keep both types of development active. For this, I recommend constantly curating different types of information sources:

• Newsletters (horizontal): for me, they are the best tool for a technology professional today. I make a continuous selection of the newsletters I subscribe to. When I receive a new publication, I select relevant links, add them to my reading list, and consume them gradually.
• Feed aggregators (horizontal): sites like Hackersnews and Lobste.rs have RSS feeds organized by theme. They work like a social network timeline, with a lot of noise, but occasionally you find useful articles. It is crucial to know how to manage high and low noise sources so that they do not interfere with the efficiency of your consumption.
• Blogs/Specialized publications (vertical): I prefer to separate these sources from traditional books because they are usually more concise, have more examples and practical exercises, and are more up-to-date than books. Examples include: https://quii.gitbook.io/learn-go-with-tests and https://www.fuzzingbook.org/.
• Papers (vertical): starting to read academic papers can be challenging, but it is an important step to know the cutting edge of development. The sooner you start, the better.
• Books and courses (vertical): I generally use them for topics that evolve more slowly, such as algorithms, mathematics, and cryptography. In these areas, you will find excellent publications that are still relevant.

Over time, as you consume content, you will accumulate links, articles, books, tools: all kinds of references. Take good care of this collection, choose a way to organize it, an app, whatever you prefer. This collection will become your garden, from which you will draw inputs to produce knowledge.

However, it is crucial not to fall into the collector’s fallacy, where we simply accumulate piles of materials and are satisfied with that.

Organizing Knowledge

One of my biggest regrets was not starting to keep organized notes earlier! My biggest recommendation here is simplicity. It’s very easy to fall into the trap of optimizing your note structure and being extremely productive (I’ve fallen several times), creating an overly complicated system.

Some things I do to structure knowledge are:

• Literature Notes: making detailed notes in your own words takes a lot of time, so I created a different type of note called Literature Note. In it, I deposit relevant information from the source of information, sometimes copying directly, other times rewriting.
  • AI Assistant: although the goal is to produce simpler and quicker notes, AI has helped me produce literature notes with the quality of subject notes. I use a prompt that generates a list of questions about the topics covered in the source of information. This has made my literature notes much more efficient. The prompt I use is publicly available: thought_provoking prompt.
• Subject Notes (Subject/Evergreen notes): these notes are the main character. They have a long life, are updated whenever I learn something new about the subject, and are written in my own words. The most important technique for me here is writing as if I were explaining to someone else.

Applying Your Method

There are still unique challenges during the course of your studies. A recurring question is: I know which subject I want to study, I know what type of study (horizontal or vertical) and I have a curation of sources - how to choose which material to consume? In this case, my method is:

1. I select material related to the theme that I may have already stored;
2. I use DORKs to search for material in the newsletters’s archives and feed aggregators.
3. Then, I dedicate some time navigating among the selected materials, until I find one (or a few) with a good balance between didactic approach and subject coverage.

This ends up leading to a slower start, but in my experience, it pays off, as it gives me the opportunity to select higher quality study materials.

Another common challenge is: how to ask questions about the subject? Currently, LLMs have been of great help to me (as long as you know how to deal with possible hallucinations). Searching for dedicated communities on Reddit often yields good results. However, the best tool is your own collection of materials: look for other materials that may explain the same concept from different perspectives. Eventually, you will put together a puzzle, gaining a much more comprehensive understanding of the subject.

Conclusion

The learning journey is deeply personal and varies from individual to individual. The strategies and methods I have shared reflect my experiences and personal preferences. It’s important to find a balance between absorbing information and creating knowledge, avoiding the trap of just accumulating references.

Remember, the key to effective study is not only in selecting the right materials but also in how you organize and interact with these materials over time. I hope this guide serves as a starting point for you to develop a study method that is as unique and effective as your own learning aspirations and needs.

This is my extension to the problems and recommendations presented on the paper Dos and Don’ts of Machine Learning in Computer Security (Quiring, et al, 2022). I encourage you to also read the paper, as it’s excellent and provide a lot of insights about how to better build machine learning models.

Machine learning workflow

Data collection

Pitfalls: Sampling Bias, Label Inaccuracy

As with any other domain, Security is also heavily affected by bad data quality used for training. However, it’s often worse in Security because data acquisition of adversary activity is usually hard and the method’s used to acquire it will bring a bias.

Let’s take for example a honeypot implemented with a vulnerable Apache Server. Even though there are lots of bad actors in the wild, if you have a medium size environment, the volume of data they will produce attacking your honeypot will not come near to the volume of packets in your production network. Also, the TTPs you are going to see will come from threat actors that are used to leverage Apache Server in their kill chain, possibly leaving other adversaries, that may be focusing on other types of vectors, producing different kill chains, out of the dataset.

On top of that, we are usually dealing with lots of examples. So, if we have label inaccuracies, dealing with them is a lot harder. We can’t just apply common techniques like using the mode to fill in, because in the Security domain, the adversary is actively trying to mimic the distribution of the benign cases. Therefore, we have very little space for noise, as it would blur even more the distinction between the outcomes.

Besides the recommendations presented in the article, I would add two more:

• User open and disseminate datasets whenever possible. Dataset sharing is still uncommon on the Security community. We already have some network and host datasets, but there are many more assets nowadays (logs for cloud, kubernetes, CDN, CI/CD, etc), and the technologies on networks and hosts are constantly evolving, posing the need for these datasets to be always updated.
• Use a model design that depends less on adversary data, such as we discussed in the previous article. This will reduce the dependency on the adversary behavior and increase the sources of data available to use.

Model design and implementation

Pitfalls: Data Snooping, Spurious Correlations, Biased Parameter Selection

Developing machine learning models is no easy task. Feature engineering, hyperparameters optimization, data preparation and appropriate splitting for validation (to avoid snooping). These are just some of the challenges when taking on this endeavor.

Using tools like AutoML may help reduce the burden, however it won’t take care of everything. In the end, you still need to understand your data characteristics and how it’s related to your problem.

But, some of these relations and characteristics tend to repeat. Time relations, for example, are extremely common and important in a lot of Security problems. Therefore, I suggest that every Security team doing machine learning on its own to think about how they can extract such aspects into reusable components.

This has the added benefit to scale the impact of the Security members that are more focused on implementing machine learning. Our domain has many different areas and enabling other teams and specialists to more easily and correctly implement models, even as proofs of concept, expands the possibilities of what the enterprise can achieve.

In the end, the best strategy to best tackle all these challenges and be more prepared to handle pitfalls it to avoid jumping to complex and sexy algorithms from the start, such as neural networks.

Using explanation techniques, as the article suggests, can help you catch spurious correlation, but even better is to use simpler and more understandable methods. I have been seen great results with simple probabilistic methods, such as with Histogram Based Outlier Score. Using a simple algorithm, we can immediately see what is driving its decision and catch faster these types of pitfalls.

Performance evaluation

Pitfalls: Inappropriate Baseline, Inappropriate Performance Measures, Base Rate Fallacy

As I mentioned before, most times when building a learning-based system in Security, we end up with imbalanced and noisy datasets. This demands a special care when choosing performance metrics to best reflect our model. Building on the previous suggestion, having common components that use by default metrics more fit to most problems in Security, such as precision, recall, and MCC, would reduce the change of human error.

However, even after computing correct performance metrics, we still need good baselines to compare them against. The article suggest goods options such as using simple methods or automated machine learning. I would add that comparing it with vendor products can also be a good experiments to fully understand the impact of replacing the third party with the model.

Operation

Pitfalls: Lab-Only Evaluation, Inappropriate Threat Model

Although the article identifies Lab-Only Evaluation as a common pitfall in Security model, I think this is one of the topic where in Security we have the most options to address.

First, we could use adversary emulation frameworks and popular attack toolkits to launch real world attacks against a test environment and study the model response. Another option would be to set up a honeypot and evaluate how well the model performed by comparing to a human analyses of the evidences afterward.

This type of experiment would also feed back into our threat model. Given the hype of AI in latest months, more knowledge is beings shared and produced about security best practices and process for machine learning, however we can still consider the topic in its infancy.

One special vector that we are still starting to discuss is supply-chain attacks to models, specially with the popularization of transfer-learning for more complex algorithms such as LLMs.

Splunk showed that more than 80% of HuggingFace’s models use pickle-serialized code, which is vulnerable to arbitrary code execution and code injection, although it’s not possible to say if any of them is malicious.

But, besides low-level vulnerabilities like this, a transfer-learning based model can also inherit the biases (intentionally placed or not) from the original model. That is, if the original model is vulnerable to an adversarial example, there is a high risk that your new model is going to also be.

This opens the possible for two types of attacks:

1. Malicious models: base models shared with adversarial examples trained, such as a large batch of images with a negative label having a red square, making the model learn that any image with a red square should be classified as negative, therefore implanting a bypass. This type of attack would be extremely difficult to detect.
2. Cross model generalization: as Szegedy, et all points in their paper, different models may be susceptible to the same adversarial examples, even when having different hyperparameters. In this case, the models would not even be so different, therefore an adversary could search for adversarial examples against the original model and just use them in the target system, with a high chance of success.

Conclusion

In conclusion, implementing learning-based systems in the security domain presents challenges that require careful consideration. Addressing issues related to data collection, model design and implementation, performance evaluation, and operational considerations is crucial. By implementing these recommendations and reflecting on these problems, organizations can enhance their capabilities in security.

To learn this function, we use a set of example data points composed of inputs (also called features) and outcomes (sometimes called labels). These example data points were sampled from the real world behavior, i.e. from the function $f$, at some time in the past. Our goal is that we can extract knowledge from the past to figure out this behavior on unseen data beforehand. This knowledge extracted from the set of examples is materialized on the $h$ function.

In Security, we can imagine some examples where this would be useful, like trying to learn if an HTTP request contains malicious payload or if some set of bytes is a malware or not. However, supervised learning is less often used in Security than in many other domains.

This happens because the principles of the supervised learning theory conflicts with the nature of Security, limiting its application. But, by understanding these principles, it is also possible to see how to best apply this technique and how it can maybe useful.

Stationary assumption

The most important principle in supervised learning is the stationary assumption. When the data that represents the real world behaviors follows the stationary assumption, it means that predicting the behavior using past example is approximately correctly.

The stationary assumption states that the behavior that is being learned don’t change through time. This has some important consequences:

1. We expect that each data point is independent of each other. This is important, because if there were causal effects between data points, then the features of a data point $x_1$, caused by $x_0$, would vary with a probability that is a combination of the probability distribution and the effect of $x_0$, hence the distribution would not remain the same over time, because $x_0$ and $x_1$ would vary in different ways. For example, in a box with 1 blue ball and 2 red balls, the probability of picking a blue ball when drawing the first one from the box is 33% while a red one would be 66%. However, if the first one is indeed blue, then the probability of drawing a red ball as the second one is 100%. The first data point (blue ball draw) changed the probability of the second data point.
2. We expect that each data point is identically distributed, i.e. each data point should be drawn from the same probability distribution. We could learn the shopping behavior using data ranging from Black Friday to New Years, however the users’ behavior in this time is completed different from the rest of the year, therefore the data used to learn has a different probability distribution than the unseen data on which we are going to make predictions.

Any dataset that follows these two characteristics is said to hold the i.i.d assumption (independent and identically distributed). The importance for our training datasets on supervised learning to be i.i.d is because it connects the past to the future, without it, any inference made on the available data would be invalid.

Understand the i.i.d assumption is specially important for Security Machine Learning because it is one of the areas where causality and behavior shifts are most present. So, exactly how this assumption affects our ability to do Supervised Machine Learning?

Causality

Many threat behaviors have causal nature, and therefore we should have a lot of care when preparing our datasets and choosing our validation methods.

A good example is malware classification, where you could have many samples from various families from different years. Each family generation influences each other, and sometimes they have similar characteristics. A special bad situation that could happen with this is that during splitting of the dataset between training and testing, without taking into account the time relation of the families and samples, then you could end up training the model with future information and testing against past samples. This would produce falsely accurate results that would not generalize in the real-world.

There are ways to deal with this, but it depends on the type and strength of the causal relation between the data. For this case, ensuring that the newest malware samples are used for test should be enough.

Behavior Shift

We could create a model to learn a threat behavior like the profile of a botnet, however once we started responding effectively to it, adversaries would adapt and our model would become useless because the new botnets would have a totally different behavior. This would be the case of a change in the probability distribution from which the features are drawn, leading to the break of our assumption.

Looking to the other side

Although causality may be addressed by good data preparation, preventing a model to be become outdated due to behavioral shift is almost impossible. However, this problem isn’t a new one in Security, such that one best practices is to instead of trying to detect and block malicious action, defenders should define what a legitimate system behavior looks like and block everything else. This can be summarized as: allow lists are more secure than block lists.

We can apply the same philosophy for supervised learning, by modeling profiles of legitimate behavior, which usually are more stable. Then, new data points are classified against these multiple profiles models, finally a meta-classifier is used to choose which one is the best fit or if it’s an outlier. This has the ability to catch any new threat behavior that deviates from the know legitimate profiles.

This combination of multiple models is called ensemble learning, which has shown to improve models performance, like when comparing a Decision Tree model to a Random Forest one.

As a side bonus, since this way of building models does not depend on knowing threat behaviors, it avoids the common problem in mining data for Security that usually produce highly unbalanced datasets. We can train each profile using the true positive data of others profiles as false examples, assuming each profile is mutually exclusive.

The main challenge in this approach is that adversaries may try to mimic legitimate profiles, however like the authors of Notos showed, this can be useful sometimes, as in their cases this would imply in adversaries using a more stable network infrastructure that would be easily defeated by static block lists. Therefore, forcing adversaries to mimic a legitimate profile could also reduce their capabilities.

Conclusion

In conclusion, supervised learning is a powerful technique that allows us to extract knowledge from past data points to predict future behavior. However, in the Security domain, applying it comes with unique challenges due to the presence of causality and behavior shifts. Understanding the stationary assumption and the importance of having an i.i.d dataset is crucial, because it informs us how to best prepare our datasets, choose our validation methods and, most importantly, what are the best behaviors to be modeled using supervised learning.

Luckily, by modeling profiles of legitimate behavior, and catching new threat behavior that deviates from known legitimate profiles, we can build intelligent allow lists.

While there are challenges, with proper preparation and understanding, supervised learning can be a valuable tool in Security.

References

• Notos: Building a Dynamic Reputation System for DNS

A common requirement that I face on multiple projects is to safeguard some API endpoints to administrative access, or to provide a secure way for other applications to consume our service in a controlled and traceable manner.

The usual solution for it is API Keys, a simple and effective authorization control mechanism that we can implement with a few lines of code. However, when doing, so we also need to be aware of threats and possible attacks that we may suffer, specially due to the usual privileges that these keys provides.

Therefore, we are going to analyze common points of concern and design a solution that improve our security posture while keeping it simple.

API Keys threats

There are two main concerns when implementing an API Key authorization scheme: key provisioning and timing attacks. Let’s review each threat before designing solutions to address them.

Key Provisioning

The key storage is directly related to how applications expect these secrets to be provided to them. Environment variables are the most common solution used on modern services since they are widely supported and don’t incur a high reading cost (in contrast to files) allowing for dynamic changes to be easily detected.

However, the way developers usually define the environment variables are through scripts or configuration files, for example using a Kubernetes Secret manifest. This introduces a serious threat of API Keys being committed to git repositories, which in the event of data leakage from the internal VCS management system would expose these credentials.

Note: remember that once committed, even if the keys are deleted from the source files, the information is already on the repository history and is easily searchable with tools like TruffleHog.

Therefore, please do not commit your API Keys to git!

Timing Attacks

Once your application is configured with the available API Keys, you need to verify that the end-user provided key (let’s call this the user key) is correct. Doing so with a naive algorithm, like using == operator, will make the verification end on the first incorrect character, hence reducing the time taken to respond.

A timing attack takes advantage of this scenario by trying to guess the correct characters of a secret based on how long the application took to respond. If the guess is right, the response will take slightly longer than if it’s wrong.

Naturally, since equality checks are orders of magnitude faster than the network roundtrip, this type of attack is extremely difficult to perform because it depends on a statistical analysis of many response samples. By looking at the time distribution produced by two different characters, one can infer that if they are different, inferring that the greater one is the correct value. For an extensive discussion of statistical techniques to help perform this attack see Morgan, Morgan 2015.

Middleware design and implementation

Having these threats in mind, we can design a suitable solution. Let’s start with the most simple API Key middleware implementation possible and iterate from it.

func ApiKeyMiddleware(cfg conf.Config, logger logging.Logger) func(handler http.Handler) http.Handler {
	apiKeyHeader := cfg.APIKeyHeader // string
	apiKeys := cfg.APIKeys // map[string]string

	reverseKeyIndex := make(map[string]string)
	for name, key := apiKeys {
		reverseKeyIndex[key] = name
	}

	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			apiKey, err := bearerToken(r, apiKeyHeader)
			if err != nil {
				logger.Errorw("request failed API key authentication", "error", err)
				RespondError(w, http.StatusUnauthorized, "invalid API key")
				return
			}

			_, found := reverseKeyIndex[apiKey]
			if !found {
				hostIP, _, err := net.SplitHostPort(r.RemoteAddr)
				if err != nil {
					logger.Errorw("failed to parse remote address", "error", err)
					hostIP = r.RemoteAddr
				}
				logger.Errorw("no matching API key found", "remoteIP", hostIP)

				RespondError(w, http.StatusUnauthorized, "invalid api key")
				return
			}


			next.ServeHTTP(w, r)
		})
	}
}


// bearerToken extracts the content from the header, striping the Bearer prefix
func bearerToken(r *http.Request, header string) (string, error) {
	rawToken := r.Header.Get(header)
	pieces := strings.SplitN(rawToken, " ", 2)

	if len(pieces) < 2 {
		return "", errors.New("token with incorrect bearer format")
	}

	token := strings.TrimSpace(pieces[1])

	return token, nil
}

A middleware is a function that takes an http.Handler and returns an http.Handler. In this code, the function ApiKeyMiddleware is a factory that creates an instance of the middleware with the provided configuration and logger. The config.Config is a struct populated from environment variables and logging.Logger is an interface that can be implemented using any logging library or the standard library. You could pass only the header and map of keys, but for clarity we choose to denote the dependency from this middleware to the configuration.

After extracting the fields that it relies on, the function creates a reverse index of the API Keys, which is originally a map from a key id/name to the key value. Using this reverse index it’s trivial to verify if the user key is valid by doing a map lookup on line 18.

However, this approach expects the API Keys as plaintext values and is susceptible to timing attacks, because its validation algorithm is not constant time.

Using key hashes for validation

To improve the key provisioning workflow, we can use a simple yet effective solution: expect the available keys to be hashes. Using this approach we can now commit our key hashes to our repository because even in the event of a data leak they could not be reversed to their original value.

Let’s use the SHA256 hashing algorithm to encode our keys. For example, if one of them is 123456789 (please, do not use a key like this :D) then its hash will be:

15e2b0d3c33891ebb0f1ef609ec419420c20e320ce94c65fbc8c3312448eb225

Now you can add this hash to your deployment script, Kubernetes Secret, etc., and commit it with peace of mind.

Next, we need to handle this new format on our middleware. This is what the code will look like now:

func ApiKeyMiddleware(cfg conf.Config, logger logging.Logger) func(handler http.Handler) http.Handler {
	apiKeyHeader := cfg.APIKeyHeader // string
	apiKeys := cfg.APIKeys // map[string]string

	reverseKeyIndex := make(map[string]string)
	for name, key := apiKeys {
		reverseKeyIndex[key] = name
	}

	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			apiKey, err := bearerToken(r, apiKeyHeader)
			if err != nil {
				logger.Errorw("request failed API key authentication", "error", err)
				RespondError(w, http.StatusUnauthorized, "invalid API key")
				return
			}

			_, ok := apiKeyIsValid(apiKey, reverseKeyIndex)
			if !ok {
				hostIP, _, err := net.SplitHostPort(r.RemoteAddr)
				if err != nil {
					logger.Errorw("failed to parse remote address", "error", err)
					hostIP = r.RemoteAddr
				}
				logger.Errorw("no matching API key found", "remoteIP", hostIP)

				RespondError(w, http.StatusUnauthorized, "invalid api key")
				return
			}


			next.ServeHTTP(w, r)
		})
	}
}

// apiKeyIsValid checks if the given API key is valid and returns the principal if it is.
func apiKeyIsValid(rawKey string, availableKeys map[string][]byte) (string, bool) {
	hash := sha256.Sum256([]byte(rawKey))
	key := string(hash[:])

	name, found := reverseKeyIndex[apiKey]

	return name, found
}


// bearerToken function omitted..

Here we extracted the logic to validate the key into a function that, before checking the equality of the user key against the available ones, encodes the user key using the same SHA256 algorithm.

This simple step improved a lot our security posture without adding much complexity. Now we can have the benefits of version control, like change history and easy detection when someone changes a key hash.

This approach works well when there are few keys to be managed, and you want to follow a GitOps approach. However, if you need to scale the key management, allow for self-service key requests and automatic rotation, you may want to look for a solution like Hashicorp Vault. Even using an external secret store I still believe this strategy, to rely on key hashes to be valid, because your external secret store can persist both the original key and the hash, and the access policy for the application can have fewer privileges in such a way that it can only read the hashes.

Constant time key verification

Once we have a better strategy to provision our keys, we need to defend ourselves against them being exfiltrated by timing attacks. The solution for this kind of vulnerability is to use an algorithm that takes the same time to produce a result whether the keys are equal or not. This is called a constant time comparison, and the Go Standard Library offers us an implementation in the crypto/subtle package that is perfect to solve most of our problems. Hence, we can update our code to use this package:

func ApiKeyMiddleware(cfg conf.Config, logger logging.Logger) (func(handler http.Handler) http.Handler, error) {
	apiKeyHeader := cfg.APIKeyHeader
	apiKeys := cfg.APIKeys
	apiKeyMaxLen := cfg.APIKeyMaxLen

	decodedAPIKeys := make(map[string][]byte)
	for name, value := range apiKeys {
		decodedKey, err := hex.DecodeString(value)
		if err != nil {
			return nil, err
		}

		decodedAPIKeys[name] = decodedKey
	}

	return func(next http.Handler) http.Handler {
		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			ctx := r.Context()

			apiKey, err := bearerToken(r, apiKeyHeader)
			if err != nil {
				logger.Errorw("request failed API key authentication", "error", err)
				RespondError(w, http.StatusUnauthorized, "invalid API key")
				return
			}

			if _, ok := apiKeyIsValid(apiKey, decodedAPIKeys); !ok {
				 hostIP, _, err := net.SplitHostPort(r.RemoteAddr)
					if err != nil {
						logger.Errorw("failed to parse remote address", "error", err)
						hostIP = r.RemoteAddr
					}
					logger.Errorw("no matching API key found", "remoteIP", hostIP)

					RespondError(w, http.StatusUnauthorized, "invalid api key")
					return
			}

			next.ServeHTTP(w, r.WithContext(ctx))
		})
	}, nil
}

// apiKeyIsValid checks if the given API key is valid and returns the principal if it is.
func apiKeyIsValid(rawKey string, availableKeys map[string][]byte) (string, bool) {
	hash := sha256.Sum256([]byte(rawKey))
	key := hash[:]

	for name, value := range availableKeys {
		contentEqual := subtle.ConstantTimeCompare(value, key) == 1

		if contentEqual {
			return name, true
		}
	}

	return "", false
}

// bearerToken function omitted...

Now, the function apiKeyIsValid uses subtle.ConstantTimeCompare to verify the user key against each available key. Since subtle.ConstantTimeCompare operates upon byte slices we don’t cast our hash to string anymore and also our reversed index has gone in place of a decoded map.

The decoding is necessary because the string representation of our key hashes are actually a hexadecimal encoding of the binary value. Hence, we cannot just cast the string to byte slice because Go assumes all strings to be UTF-8 encoded.

Note: for an example on how using a cast instead of the correct decoding function, the result of []byte("09") is 110000111001 while hex.DecodeString("09") produces 1001. Check out the live example here.

The major disadvantage of this solution is that now we need to iterate over all available keys before finding out if the key is incorrect. This doesn’t scale well if there are too many keys, however one simple workaround would be to require the client to send an extra header with the key ID/name, e.g. X-App-Key-ID, with which you can find the key in O(1) and then apply the constant time comparison.

However, there is one subtle (pun intended) behavior from subtle.ConstantTimeCompare that we must be aware before deploying our solution to production. When the byte slices have different lengths, the functions returns earlier without performing the bitwise operations. This is natural because it does an XOR between each pair of bits from each slice, and with slices of different sizes, there would be bits from one slice without a matching pair to be combined with. Because of it, an adversary could measure that keys with the wrong length have a smaller response time than keys with the correct length, hence leaking the key length. It would only be a vulnerability if you use a short key that is easily brute-forced, but with a simple 30 character key using the UTF-8 printable characters you would have 30^95 = 2.12089515 × 10^140 possible keys.

Finally, we’ve built a simple, secure and efficient API Key solution that should handle a lot of uses cases without additional infrastructure or complexity. Using a basic understanding of threats and the Golang standard library, we could do a security-oriented design instead of leaving security as an after-though in an iterative way.

Photo by Silas Köhler on Unsplash.