Architecture
Bitrix24 integrations: queues, retries and duplicate prevention
What to consider when an external request times out, its response is lost or the same event arrives more than once.
A timeout does not prove that an external operation failed. The service may have saved the data while its response was lost in transit. Retrying a creation request can produce a duplicate. Start by defining how to identify operations and recover from failures.
The following is a design approach for enterprise integrations. It does not describe a built-in Bitrix24 guarantee: implementation depends on the APIs on both sides.
Establish data ownership first
For every entity, identify the source of truth and which fields the other side may change. Without this rule, two-way exchange quickly becomes a conflict between automations overwriting each other's data.
Maintain an identifier mapping: source, external ID and local ID. Each operation also needs a stable key that survives redelivery. A new random key on every attempt cannot identify a retry.
What idempotency means
Repeating idempotent processing for the same operation does not produce an additional unwanted effect. For example, a repeated request-registration event should refer to the task already created, rather than create another one.
Checking that a record does not yet exist before creating it is insufficient under concurrency: two handlers may check simultaneously. Consistent storage-level protection or the receiving API's idempotency mechanism is needed.
If the external system does not support an operation key, design another reconciliation method. Sometimes this means searching by an agreed external identifier; sometimes an uncertain result needs manual investigation. Universal exactly-once delivery across arbitrary systems cannot be promised.
Why a queue is useful
A queue separates the user's action from the exchange attempt and stores its state. It must survive process restarts and make unfinished work visible.
Example record fields include operation ID, event type, entity reference, data version, state, attempt count, next-attempt time and last error code. Choose the payload based on what recovery requires. Exclude credentials and unnecessary personal data.
| State | Meaning |
|---|---|
| Pending | The operation is accepted for processing |
| In progress | A specific handler has claimed the attempt |
| Retry required | A temporary error has been confirmed |
| Completed | The expected result was received and saved |
| Investigation required | Automatic retries are unsafe or exhausted |
Claiming a record must prevent simultaneous execution by two handlers. Recovery is also needed if a handler crashes. Reclaiming a job is safe only when duplicate effects are prevented as well.
Distinguish error types
For temporary outages, schedule the next attempt with increasing delay and a small random variation. This reduces the chance of all handlers retrying the service simultaneously.
Invalid data formats, missing permissions and business conflicts require fixing the cause. Retrying without changes is pointless. After a request has been sent, a timeout may provide insufficient evidence to distinguish failure from successful execution without a response.
For Bitrix24 REST, account for documented errors and limits. Cloud constraints should not automatically be applied to self-hosted installations, where server and module settings matter. Check numerical limits against current documentation and the actual environment.
Consistency and event ordering
If a database change is saved but the queue entry is lost, exchange never starts. When both records use the same transactional database, consider saving the job and data change in one transaction: the outbox approach. Separate stores require their own recovery and reconciliation strategy.
Late events are another issue. An old change should not overwrite a newer one merely because it arrives later. Use an agreed entity version or another ordering mechanism supported by both sides.
Minimum verification scenarios
- Deliver the same operation twice: no unwanted duplicate appears.
- Run two handlers: they do not execute the same record simultaneously.
- Interrupt the response after the external save: retrying does not create another object.
- Restart a handler: unfinished work is detected and recovered.
- Deliver an old change after a new one: the current state is preserved.
- Send invalid data: the job receives an understandable error and does not loop indefinitely.
Track the age of the oldest job, error counts, queue states and log entries linked to specific operations. These signals complement the checks above. A running handler alone does not prove that data exchange works correctly.