API Heuristic Checklist – Common Falsehoods

I found yet another wonderful resource by Falsehoods. It’s on GitHub. kdeldycke/awesome-falsehood: 😱 Falsehoods Programmers Believe in (github.com)

Pro Tip: The webpage will save the status of the checklist until you reset the page cache or a timeout happens (180 days). So, you can use this as a workbook too.

API Falsehood Checklist

Does the API assume that developer knows something about app?

Jargons on API vs UI Layer? Are they same vs different?

Does API documentation has list of App Jargons Used? Abbreviations?

Is the documentation of old versions preserved?

Are you forcing users to upgrade? What if they don't want to?

Does the document Differentiate Similar Sounding Terms? (Ex: Tags vs Topics vs Terms)

What if 200 Response Code with Error in Response Message? Silent Crash Scenario?

Rate Limit? Is it documented? Is there a code for it?

Is OAuth Lib reliable? What if it doesn't work someday? Service agreements?

Breaking Changes only in Explicit (or Major) Versions?

No feature disappears? or deleted without going in deprecated mode?

Backwards incompatible changes announced long before they are deployed.

We don't just update the docs and pretend the old spec (features) never existed. Facility for archive.

If there is an announcement of breaking changes, it will explicitly call out the changes. changed. It won't just be a dump of the entire new documentation.

Acknowledging customer if the bug is going to be fixed.

You are not removing features because someone raised bugs on it without acknowledging it.

A channel to report bugs made clear to customers. Github? Forms? Email?

Happy Testing!

Checkpoint Explanations

1. Does the API assume that the developer knows something about the app? This refers to whether the API documentation assumes that the developer has prior knowledge about the application or system for which the API is being used. API documentation should be written in a way that is easy to understand for beginners, without assuming any prior knowledge about the application or system. This helps to ensure that developers who are new to the technology or application can use the API effectively.

2. Jargons on API vs UI Layer? Are they the same vs different? This refers to whether the jargon or technical terminology used in the API documentation is consistent with that used in the user interface (UI) layer of the application or system. Consistency in the use of technical terminology enhances the clarity and understandability of the API documentation, making it easier for developers to use and integrate the API into their applications.

3. Does the API documentation have a list of app jargon used? Abbreviations? This refers to whether the API documentation includes a list of jargon or technical terms used in the application or system, along with their explanations or abbreviations. This improves the clarity and understandability of the API documentation, making it easier for developers to understand and use the API effectively.

4. Is the documentation of old versions preserved? This refers to whether the documentation of older versions of the API is preserved and made accessible to developers, even after newer versions of the API are released. Preserving documentation of older versions of the API is important for maintaining backward compatibility and supporting developers who may still be using older versions of the API.

5. Are you forcing users to upgrade? What if they don’t want to? This refers to whether users of the API are forced to upgrade to newer versions of the API, even if they do not want to or are not ready to do so. Testing this falsehood helps to ensure that users of the API are not forced to upgrade to newer versions against their will. This ensures that users have the flexibility to choose when and if they want to upgrade, based on their own needs and timelines. It helps to maintain a positive user experience and avoid potential disruptions caused by forced upgrades.

6. Does the documentation differentiate similar-sounding terms? (Ex: Tags vs Topics vs Terms) This refers to whether the API documentation clearly differentiates between similar-sounding terms or concepts used in the API. Testing this falsehood improves the clarity and accuracy of the documentation, making it easier for developers to understand and use the API effectively without confusion. It helps to prevent misunderstandings and potential errors in API usage, leading to better product quality and developer experience.

7. What if a 200 response code with an error in the response message? Silent crash scenario? This refers to the scenario where the API returns a 200 response code, which typically indicates a successful request, but the response message contains an error. This can result in a silent crash scenario where the error is not explicitly communicated to the developer, leading to potential issues or errors in the application or system using the API.

8. Rate Limit? Is it documented? Is there a code for it? This refers to whether the API documentation includes information about rate limits, which are limits on the number of requests that can be made to the API within a certain time period. Properly documented and enforced rate limits help to manage API usage, prevent abuse, and ensure fair usage of resources, leading to improved API performance and stability.

9. Is OAuth Lib reliable? What if it doesn’t work someday? Service agreements? This refers to the reliability of the OAuth library used in the API for authentication and authorization. OAuth is a widely used standard for secure API authentication and authorization, and it is important to ensure that the OAuth library used in the API is reliable and secure. Reliable and secure authentication and authorization mechanisms are critical for protecting sensitive data and ensuring the security of API interactions, which ultimately enhances the trust and reliability of the product.

10. Breaking changes only in explicit (or major) versions? This refers to whether breaking changes, which are changes that can potentially break the existing functionality of the API, are only introduced in explicit or major versions of the API. Breaking changes can have a significant impact on the applications or systems using the API, and it is important to manage them carefully to avoid disruptions and compatibility issues. Ensures that developers have sufficient notice and time to prepare for and handle any necessary changes in their applications or systems before the API-breaking changes are implemented.

11. No feature disappears or is deleted without going into a deprecated mode? This refers to whether features in the API are deprecated before they are removed or deleted. Deprecation is a process of marking a feature as obsolete or outdated, typically accompanied by a timeline for its removal, to provide developers with a transition period to update their code before the feature is actually removed or deleted from the API.

12. Backward incompatible changes are announced long before they are deployed. This refers to whether backward incompatible changes, which are changes that are not compatible with previous versions of the API, are announced to developers well in advance before they are actually deployed. Providing advance notice of such changes allows developers to prepare and update their code accordingly, ensuring smooth transitions between API versions.

13. We don’t just update the docs and pretend the old spec (features) never existed. Facility for an archive. This refers to whether the API documentation preserves information about old versions or deprecated features, rather than simply updating the documentation and disregarding the existence of previous versions or deprecated features. Preserving information about old versions or deprecated features in an archive allows developers to refer back to them for historical or reference purposes.

14. If there is an announcement of breaking changes, it will explicitly call out the changes. It won’t just be a dump of the entire new documentation. This refers to whether announcements of breaking changes in the API explicitly specify the changes that are being made, rather than providing a dump of the entire new documentation. Explicitly calling out the changes provides clarity and allows developers to easily identify and understand the specific modifications being made to the API.

15. Acknowledging customers if the bug is going to be fixed. This refers to whether the API provider acknowledges customers when a reported bug is going to be fixed, providing feedback and updates on the status of bug fixes. Acknowledging customers’ concerns and keeping them informed about the progress of bug fixes helps to build trust and maintain good customer relations.

16. You are not removing features because someone raised bugs on it without acknowledging it. This refers to whether the API provider removes features solely based on reported bugs without acknowledging or investigating the reported issues thoroughly. Proper acknowledgement and resolution of reported issues contribute to improved product quality, user satisfaction, and trust in the API provider.

17. A channel to report bugs made clear to customers. Github? Forms? Email? This refers to whether the API provider has a clear channel for customers to report bugs or issues they encounter while using the API. Providing a clear and accessible channel for reporting bugs allows users to provide feedback, report issues, and contribute to the improvement of the API.

I would also recommend you to check out another checklist for security testing by Noname Security which can be found via this link: API Security Best Practices & Checklist

In conclusion, testing these common API falsehoods is essential to ensure that the API provider follows pleasing practices in terms of documentation, communication, customer support, and maintaining backward compatibility. By addressing these areas, API providers can improve the quality, reliability, and user satisfaction of their products, and build trust with their developer community.

If you liked this article, you can check out this popular post from my blog hits: