Best practices
This guide contains best practices for designing and running flows.
Designing flows
Keep your messages clean
Within a flow a message can grow in size, because of added headers and data in the body. It is good practice to clean up your messages as much as possible, especially before communicating with endpoints.
- Use the Remove Headers component to clean up headers you are not using anymore.
- Clean up the body whenever it is not applicable anymore, for example with an empty JSON (
{}
) or XML (<root/>
).
Use variables
It's important to understand the different kinds of variables within Dovetail and how to use them to your advantage. Read more about Using variables in Dovetail.
Flow properties
It is good practice to use Flow properties for efficient configuration of settings in your flow, for example the base url of an API. Using Flow properties improves the scalability of your integration solution, making it much easier to copy similar solutions to different clients with only minor edits.
Tenant variables
It is good practice to use Tenant variables for data that is used in multiple flows or to change the functionality of a flow without having to edit and reinstall it.
Data formats
In application integration you encounter a big variety in data formats. It is good practice to:
- Use standardised data formats like XML or JSON over other formats like CSV, Excel, ASCII.
- Use text based data formats, instead of binary.
- Keep the size of messages lower than 10 MB.
Protocols
In general it's recommend to use HTTPS endpoints over other endpoint protocols like SFTP, email (SMTP) or file. HTTPS uses internet standards and scales well. For scalability also a RabbitMQ endpoint is provided.
Document your flows
Use the Notes, Component notes and Version remark to document how your flow works. It will greatly improve your understanding of a flow, especially over time. Your future self thanks you in advance!
Flow versions
Limit the number of Flow Versions to maintain application responsiveness and delete old versions when possible.
Encoding
When working with text data, it's essential to understand the encoding being used to ensure that characters are interpreted correctly. UTF-8 has become the standard for web content due to its versatility and compatibility with ASCII. Other encodings are still relevant in certain contexts, particularly for legacy systems and specific language requirements.
It is good practice to use the Encoder component to transform the encoding of a message to UTF-8 in flows for optimal compatibility with Dovetail components.
Pretty vs Raw HTTP responses
Be mindful that tools like Postman may alter responses for readability, possibly changing the encoding and/or data format layout 'under the hood'. Ensure you are viewing 'raw' responses for accuracy. Keep in mind that this 'raw' response is the response that Dovetail gets.
Scripts
It is good practice to minimise the use of the Script component. While it gives a lot of flexibility it also:
- Makes it more difficult or impossible for us to support your integration solution.
- Takes more resources on the server.
- Updates to the component and/or it's underlying libraries can impact on how the Script functions. Leading to:
- Testing, and possibly extra development, when Dovetail updates.
- No guarantees regarding backwards and/or forwards compatibility of Script code between Dovetail versions.
Running flows
Flow performance
To optimise the performance of your flow it is good practice to:
- Keep the number of steps in a flow as low as possible.
- Keep the number of linked flows as low as possible.
- Keep the number of high impact steps as low as possible.
- Prefer the Exchange pattern
One way
overRequest reply
. - Use the Transport
Synchronous
by default. Avoid using Transport typeQueues
within flows whenever possible. Messages are stored on a broker when set toQueues
, this makes processing them slower (it's used to keep memory consumption low). - Disable tracing in production (see Trace Configuration)
Trace configuration
Transactions are stored in a database on the server your Dovetail instance is running on. Avoid enabling tracing on flows that process large files and/or headers and set the time to the smallest value you can to make sure the server doesn't run out of disk space.
Keep in mind that saving a copy of each message can impact the performance of your flows. Disable tracing on flows that process large quantities of data and/or large files. Implement a Management by Exception solution to monitor these types of flows.
Exception handling
Use a Management by Exception principle and define the expected response of an endpoint. All unexpected responses can be logged and/or handled functionally from within your flow. Refer to our Management by Exception guide for an extensive tutorial on how to implement this.
Good integration solutions
To summarize it's a good practice to:
- Make integrations between two endpoints (systems) instead of multiple endpoints (Keep it simple).
- Use scalable protocols/technologies like HTTPS, JMS, AMQP over not scalable ones like SFTP, File or Email.
- Use structured dataformats (XML or JSON) of fixed small size. This makes it easy to process messages in a predictable way.
- Keep as much of the application logic outside the integration platform.