If you’re an API or application developer, you likely hear the term “token” a lot. However, that term has entirely different meanings depending on the developer and the industry.

Maybe you’re a developer who just implemented token-based authentication or found out the AI agent you built uses far too many LLM tokens. Perhaps you’re experimenting with crypto tokens, developing a game that allows users to purchase in-game items with cryptocurrencies.

The point is that there are many types of tokens, and this article covers four main categories: authentication, LLM, crypto, and miscellaneous. If you’re confused about which token does what and why they matter, this guide is for you.

Please note that some of these definitions refer to a “user.” In most cases, a user can be a human or a machine, such as an AI agent.

1. Authentication Tokens

Authentication tokens enable users to verify their identity so that they can gain or maintain access to a website or application.

There are three main types of authentication: token-based, session-based, and passwordless. There is also the Initiative for Open Authentication (OATH). All these methods of authentication involve tokens in some way.

Token-Based Authentication

When we say “token-based authentication,” we’re referring to a modern authentication method that uses a stateless, self-contained model that relies on a combination of tokens. The token state lives on the client side and does not have to be stored on the server.

Tokens used in this model include:

• Access token: Grants a user access to specific resources, such as APIs. It can be short-lived or long-lived and contains information about the user and the resource the token is intended for.
• ID token: Carries information about a user so that clients can verify and authenticate their identity. Note: An ID token serves as proof of authentication, while an access token serves as proof of authorization.
• Refresh token: Enables a user to obtain a new access token without re-authenticating once the current one expires.
• Bearer token: Grants access to the party holding the token. A bearer token is typically an opaque string, providing access to protected resources without requiring additional proof of identity. It is the predominant type of access token used with OAuth 2.0.
• JSON Web Token (JWT): An open standard (RFC 7519) for representing claims that are to be transferred between two parties. Information is securely transmitted as a compact JSON object and can be verified via digital signature and/or encrypted. Bearer tokens are often implemented as JWTs in practice.
• Opaque token: Also referred to as a “reference token,” opaque tokens contain a unique, random alphanumeric string generated by the authorization server. They carry no readable or actionable information.
• SAML token: Security Assertion Markup Language (SAML) is a protocol used primarily to enable single sign-on (SSO) in applications. SAML tokens are in XML format, containing information that confirms to the service provider the authentication of the user signing in.

Token-based authentication is widely used by application developers today. It has become popular because of its high scalability, compatibility with APIs, and the proliferation of industry standards, such as OAuth 2.0 and OpenID Connect.

Note: OAuth is a popular authorization framework, and some developers use the term “OAuth tokens,” which generally refers to any tokens in an OAuth authorization flow.

Session-Based Authentication

This is a traditional approach to verifying user identity where the token state lives on the server side, and tokens are stored on the server. This method has one main token type.

• Session token: A unique identifier that authenticates a user to a web application (or website) and allows the app to manage the user’s activity during a session. The session token is created when a user logs in. It is usually stored in a cookie or locally on the user’s device.

While token-based authentication has become a popular method for verifying user identities, session-based authentication is often adequate for many web applications.
However, in the past several years, the tech industry has started moving toward passwordless authentication, promoting methods such as passkeys, biometrics, and magic links.

Passwordless Authentication

Most forms of passwordless authentication do not involve a token at the onset for verifying the identity of the user. However, there is one that does:

• Magic link: A URL with a unique, time-limited embedded token that authenticates a user so they can log into their account. The user receives the magic link via email, SMS, or messaging app.

For all types of passwordless authentication, including passkeys and biometrics, a token is issued after authenticating. It is used by the system to manage the session going forward.

OATH Tokens

OATH is an industry-wide collaboration to define and promote open standards for strong authentication. The initiative focuses primarily on one-time passwords (OTPs) and the OATH Challenge-Response Algorithm (OCRA).

One-time passwords are not tokens per se. However, companies like Microsoft, Thales Group, and Deepnet Security now support OATH authentication tokens:

• Software OATH token: An application on an existing device — typically an authenticator app like Microsoft or Google Authenticator — that implements OATH standards to generate OTPs.
• Hardware OATH token: A physical device that generates OTPs using OATH standards. It is a dedicated piece of hardware with no other purpose. One example is OTP Display Cards.

The Initiative for Open Authentication was founded in 2004 but has been gaining support in recent years due to the need for strong authentication methods across billions of devices and applications.

Note: OATH is different from OAuth. OATH tokens specifically produce OTPs, and OAuth is an open authorization protocol.

Why Authentication Tokens Matter

Authentication tokens are a critical part of securing applications and websites because they help prevent unauthorized access.

These tokens are usually short-lived, encrypted, and specific to a user’s session, application, or device, making them difficult to reuse if stolen. They are a more secure means of verifying user identities than the traditional “user and password” only method.

People often forget their passwords or use the same password for multiple applications, making it easy for cybercriminals to brute-force hack their accounts. Many websites and applications today still use passwords or API keys but also incorporate authentication tokens for stronger security.

2. AI and LLM Tokens

For authentication, tokens are used to confirm the identity of the user. For large language models (LLMs), they are numerical representations of text. LLMs don’t see and understand text as humans do. They require text to be broken down and represented as numbers, a process referred to as tokenization.

Screenshot of Tiktokenizer, an app that shows how LLMs break down text into numbers.

LLMs often power AI agents and interact with MCP servers. This process has four main types of tokens:

• Input tokens: When you enter text into an LLM prompt, the model generates input tokens.
• Output tokens: The text the model outputs based on a prompt results in output tokens.
• Reasoning tokens: Some LLM companies provide reasoning models that use internal reasoning tokens before producing a response.
• Cache tokens: Model prompts often contain repetitive content or consistent elements which can be cached to reduce token usage. Most providers have two types of cache tokens:
  • Cache write tokens: Tokens that apply when the cache is created for the first time.
  • Cache read tokens: Tokens that apply when the cached content is retrieved. These tokens are usually less expensive than input and cache write tokens.

As more developers build AI agents and MCP servers, they will need to become familiar with the different types of LLM tokens and the purpose of each one.

Why AI and LLM Tokens Matter

LLM tokens matter because they directly impact the cost of using the model and how well it performs.

If you are developing an application that relies on an LLM, you’ll want to pay close attention to which tokens it uses and how many. Most LLMs have pricing structures based on token usage, and every interaction with the AI model generates tokens. The more tokens your application uses, the higher the cost.

Another thing to consider when it comes to token use is an LLM’s context window, which essentially serves as the AI model’s working memory. Every LLM has a context window limit typically measured in tokens, and that limit varies per model. All input and output tokens count towards this limit. Some LLMs like Claude have “extended thinking” capabilities, and the tokens used for thinking count as well.

If your application exceeds the number of tokens allowed in the context window, it can result in an error. Approaching the limit can impact response quality because the model has less room in the context window to work with.

3. Crypto Tokens

A crypto token is built on blockchain technology, so it is sometimes called a blockchain token. It is a digital representation of an asset, such as:

• Financial assets, such as currency, commodities, and company equity.
• Physical assets like real estate, precious metals, and artwork.
• Digital assets, such as music, video, games, and in-game items.
• Services like event tickets, API subscriptions, and rewards program points.

There are three main types of crypto tokens:

• Fungible token: An interchangeable digital asset that possesses equal value to others and follows a standardized set of rules. Examples: Bitcoin (BTC), stablecoin, and Ethereum (ETH).
• Non-fungible token (NFT): A token that represents a unique and non-interchangeable digital asset. Example: Beeple’s “Everydays: The First 5000 Days.”
• Semi-fungible token: A token that is initially fungible but once it loses its face value, it becomes non-fungible. Examples: ERC-404 and DN-404.

You can find many use cases for crypto tokens, including tokenizing assets like real estate and art, in-game currencies and game asset ownership, and royalties for intellectual property.

Why Crypto Tokens Matter

Crypto tokens provide developers with new ways to monetize and innovate applications. They can experiment with different reward structures and facilitate transactions, including tokenizing assets. They can build secure decentralized applications that protect user privacy. They can also use crypto tokens to encourage digital asset ownership within applications. These tokens can be used to digitally represent various real-world assets, so developers can use them to build innovative applications in a wide range of industries.

Note: Smart contracts are central to how developers implement crypto tokens in applications.

4. Application and API Tokens

Not all tokens fit neatly into the above categories. Here are a few miscellaneous types developers should know that are related to application and API development.

• CSRF token: A cross-site request forgery token is typically embedded in a web page and serves as a first line of defense against a CSRF attack. It contains an unpredictable value, making it difficult to create a forgery.
• Password reset token: Contains a single-use password generated by the server and delivered to the user through an external mechanism like email or SMS. It allows the user to reset their password if they forgot or lost the original.
• Idempotency token: A unique token included in API requests to ensure that if the same request is sent multiple times, it is only processed once. It is a common component of payment APIs.

As you can see, there are many different types of tokens that perform a wide variety of functions in applications and websites.

Tokens Are Pervasive in Software Development

And there you have it — a guide to the types of tokens that exist today. You’ve learned about authentication tokens that help secure applications and how token usage impacts the cost of using an LLM. You’ve also learned how developers can innovate applications with crypto tokens and how some tokens don’t fall neatly into a category. Staying current on token types is a critical part of building modern applications, and this guide can help get you started.