Tech Tuesday – Web API Security
13 September 2016 |
About a 12 minute read
This is a broad overview of the things to consider whilst securing web APIs – it discusses why security is often overlooked, why it’s important and the various techniques available to developers. This is not a detailed how-to guide of implementing these techniques, rather a checklist of things to use whilst designing an API.
Security is holistic: one weak link in a system can cause the security of the entire system to collapse. Therefore, the approach taken here is to consider the end-to-end API system – from authentication at the service interface to encryption of the database.
Why It’s Overlooked?
Security is often overlooked for the following reasons:
- Pressure from deadlines
- Cost cutting or limited budgets
- Complexity of execution
- Specialist skills required
- Value is not tangible upfront
Why It’s Important?
Security is important:
- To maintain data integrity and confidentiality
- To protect against data loss, theft and damage
- To ensure availability / uptime
- To retain reputation of the brand
- To keep intellectual property safe
- To avoid breaking privacy laws
- To avoid expensive court cases
Exposure of public APIs by organisations has grown exponentially in the last 10 years for access by other organisations, service providers and users.
Internal APIs should also be protected – if an attacker can get into an internal network and past a corporate firewall, then data is also at risk. With ever increasing exposure of potentially sensitive data via APIs, it is important to ensure that these APIs are secure.
Sources: Programmable Web http://www.programmableweb.com/
How Can It Be Applied?
At AND Digital we build internal products with the following properties:
- SOA / microservices architecture
- APIs on the internet
- Externally hosted: AWS
- Sensitive data about:
- Our people
- Our clients
- Commercial information
Given the above state, it is important that both at AND Digital and within your organisations, we apply the approaches in this post to ensure data remains confidential, has integrity and is available.
- Challenge our product owners to prioritise security in the backlog
- Drive our internal product teams to build secure applications and APIs
- Mentor product teams in API security practices
- Incorporate security review into our technical review process
- Ensure our technology suppliers (third parties) provide secure services
AND Digital have a wide range of clients ranging from e-commerce to banking. Our clients:
- Hold sensitive data
- Care about privacy and data confidentiality
- Care about their brand’s reputation
- Are aware of recent high-profile security breaches
- Have web APIs – microservices, REST, SOAP
Clearly, API security is highly relevant to the majority of our clients and we can advise them on this.
- Explain the idea of security implementation and testing to senior stakeholders
- Identify and communicate gaps and risks in security to stakeholders
- Challenge product owners to prioritise security in the backlog
- Demonstrate our expertise in finding solutions to security problems
- Upskill our client’s developers in API security best practices
- Incorporate security review into our clients technical review process
Guidelines Catalog & Checklist
The level of application of these techniques (all, some or none) is highly context dependent, and each will have trade-offs such that there is no perfect combination. But in most cases, some minimal level of security should be applied to any web API.
Encrypt Data In Transit (TLS)
Data that is communicated on the network is called Data in Transit. Data in Transit should be encrypted using TLS to ensure it’s protected against man-in-the-middle attacks and sniffing, where an attacker who has access to the network can intercept and inspect packet data sent across the network.
This applies to the following network communications:
- From client/user/browser to service
- From service to database
- From service to other services/infrastructure (internal and external)
It is important to identify where sensitive data is being transmitted over a network and ensure it is encrypted.
With TLS encryption times being almost negligible on modern hardware, encryption of Data in Transit should be the default approach with exceptions made only where applicable.
Signatures can be generated from the API request or responses to ensure data is not tampered during transit. By encrypting the signature, only the receiver can decrypt it and use it to validate the payload – this prevents modification of the signature and payload by an attacker. Essentially, if the signature of the received payload does not match that of the signature generated by the sender, then the payload has been tampered with. This is a useful technique if the message/payload itself can be transmitted unencrypted (e.g. non sensitive data), but you still require integrity of the transmitted data.
Encrypt Data At Rest
Data that is stored should be encrypted using the following considerations:
- Strong algorithms
- Strong encryption keys / key rotation
- Encryption down to filesystem
This is not only for production / online data storage, but also backups.
Protect Session State
In the web services world (particularly REST), services are commonly stateless as this can significantly improve server performance and scalability. Any session state is usually held on the client side e.g. in-memory, local storage or browser cookie.
It is important to encrypt auth. session data on the client side to prevent an attacker from using this data to hijack the session. A strong encryption algorithm should be used, and the keys should only be held on the server side.
This is particularly important when using something like JWT (JSON Web Tokens) to hold authorisation information on the client side. If an attacker were to decrypt the JWT due to poor encryption, he/she could alter authorisation roles to allow unrestricted access to the API.
Consider the following for encryption of auth. session data:
- Strong algorithms
- Strong keys / key rotation
Use Opaque IDs
A guessable ID such as something short, simple and sequential can mean an attacker can easily guess the next ID and use it for attacks. Instead of using “1234” consider using a UUID such as “acc770f0-2572-11e6-bdf4-0800200c9a66”, where the UUID is randomly generated and not easy to guess. Ensure the ID strategy is URL safe e.g. using Base62 encoded UUIDs, so that IDs can be used in resource URLs safely and without URL encoding.
Whilst the simplest approach for authentication, it is the least secure approach versus other common approaches like OAuth. However, it does set the basic standard for authentication and is better than no authentication at all. Usernames and passwords are Base64 encoded and sent as part of the HTTP header on each request. It must be used with TLS encryption for Data in Transit.
No Credentials In URL
Avoid any of the following in the URL:
- Session token
- API key
Placing these in a URL can mean they are captured in web server logs which exposes a security risk if an attacker were to gain access to these logs.
OAuth2 is a authorisation framework that allows a service provider/client (e.g. a mobile application for users) to gain access to a resource on the resource server (usually accessible via a HTTP REST service). This is done by allowing the resource owner (usually the end user) to authenticate and then authorise access to the service provider via an authorisation server. It is particularly useful approach if the client does not own the data they are attempting to read.
The below links are good explanations of how OAuth 2 works, and the various flow options in OAuth 2.
It is important to understand the OAuth semantics – especially OAuth 2 as it is more flexible than OAuth 1a, thus misuse could result in a security vulnerability.
These can be used in place of username and passwords where a specific application needs to be identified. It is seen as more secure as it is simply a generated identifier for an application or client service – no username/password is submitted. So a client or service provider (e.g. a mobile app) would pass API keys to the resource service to identify itself. The resource service, which issued the API key, validates that it is a trusted client. An API key may be for the end user e.g. if there is no need to uniquely authenticate/authorise the end user. However, in many cases the API key simply identifies the service provider and something like OAuth is used to authenticate and authorise the end user that owns the resource.
Authenticate Every Request
Frameworks like OAuth make this a default, but otherwise every request to the API should be authenticated i.e. the authorisation credentials (username/password, api key, auth token) should be passed in each request to the web service.
A strict form of access control is to whitelist certain users, for example by:
- IP address / netmask
- Account ID
- Device number
- Mac address
This can be a very granular authorisation level e.g. only allow user with IP address and device ID to delete accounts.
Short Sessions & Token Expiry
User sessions/auth tokens should have a short time to live to ensure that they expire often. This reduces the time window in which a token can be exploited by attackers. The web service / auth. service should verify and enforce session expiry.
Auth. frameworks like OAuth have this approach built in, but the configured expiry time should still be considered carefully on a case-by-case basis.
Protect HTTP Methods
Scope API usage to specific operations or methods on specific resources. Restrict based on the following factors:
- API / session key
- Resource collection
- Action / method
- Bob’s session
- User preferences (not user)
- DELETE (not POST or PUT)
- Only records 1 to 5, which represent a fixed set of customisable preferences
Protect Privileged Actions
Not all resources are accessible by every user e.g. administration functions. It is important to ensure these are identified, categorised and protected from unauthorised use e.g.
Remove Unnecessary Methods
Avoid creating HTTP methods that are not required or will never be used.
Some frameworks auto-generate all HTTP methods for a resource e.g. GET, POST, PUT, DELETE etc. It is important to disable or remove methods that are not needed to avoid exposing a point of attack.
Mutually Assured Certificates
This approaches ensure only clients with a certificate can access the web service. A certificate can be securely distributed to a restricted set of clients to enforce limited access to trusted organisations or users.
This greatly restricts the scope of authorisation, but is exposed if the certificate is ever made public e.g. through insecure distribution, human error.
It is particularly useful for strict or highly privileged access scenarios.
Injection Attack Prevention
It is important to protect against injection attacks where an attacker can supply data to an API which has unintended and malicious consequences.
Taking SQL injection as an example:
- If the attacker called /accounts?id=<query_parameter>
- And this directly mapped to a SQL statement: “select * from accounts where id = ” + query_parameter
- If the attacker supplied query_parameter ‘105 or 1=1′ this would return all the accounts and not just account 105.
Injection attacks can occur in a number of ways:
- SQL injection
- XPath / XML attacks
- JSON attacks
To prevent against such attacks:
- Use parameterised query APIs for database calls
- Restrict amount of data – request size
- Validate data at a business/domain level e.g. email address syntax, credit card number length, formats
- JSON/XML marshalling frameworks with validation
- JSON/XML Schema validation
- Injection detection
- Virus scanning
- Blacklist pattern recognition
- Policy enforcement
- Other content filtering
- Worse case – validate input parameters manually
Rate Limiting / Quotas
This help prevents DOS (denial of service) attacks by stopping excessive requests coming from a particular source:
- IP address
This not only help balance load from all users, but also prevents an attacker from overwhelming the web service / server. Also, consider limiting request sizes to ensure no single request can cause a DOS e.g. image upload size, bulk data size.
Use friendly and generic error messages with identifiable error codes when returning errors from an API. Never return stack traces or any details which can give an attacker details about the implementation of your system e.g. technologies. Also, do not forget to use the correct HTTP response code when returning an error.
Never log anything in the web service which is sensitive e.g. credit card numbers, passwords, names. Instead log non-meaningful identifiers that can be used for tracing. It is also a good idea to log access requests to the API as these can help identify malicious behaviour from attackers.
A web API and associated components (database, network) are going to be hosted somewhere – hardware (virtual or physical) and possibly containers. The following is a list of additional things to consider:
- Encrypt data communication – SSH
- Use non-standard ports
- OS security patches
- software libraries
- Remove unwanted software and services
- disable FTP
- disable other non-secure access
- Firewall rules / security groups
- VPN communications
- Account restrictions
- password policies
- account locking
- disable root access
- BIOS lockdown & password
- Logging and auditing enabled
- Intrusion detection systems
Senior Full Stack Developer (London)
Champion software quality and technical vision for AND and our clients, work on large-scale projects and help junior and mid developers grow in their roles.I'm Interested
Full Stack Developer (London)
Put your development expertise to work, building remarkable, digital products in Agile environments using a variety of languages and frameworks.I'm Interested
Tech Lead (Reading)
Bring your expert tech knowledge to the table to influence the direction of projects, whilst coaching and your team through engineering best practices.I'm Interested