Bug fixes
Bug
The system had a bug which caused inconsistencies in the back-pressure system that eventually lead to stuck process.
This happened when there were many executions (.run()) for the same reused Caminho instance, so when some of the operators threw an error in the execution, the back-pressure system would not decrement the pending items for the items that has generated but haven't finished all the operators, therefore, when back-pressure system compared the maxItemsFlowing with the current number of items flowing, it was still accounting for items that were not in memory anymore.
Over time, the Caminho instance was allowing less and less items to be in memory, according to the number of items that were flowing during a failed execution, so at some point, the number of "ghost" items would reach maxItemsFlowing and stop processing completely.
Fix
The back-pressure system currently accounts all runs for a given Caminho instance, this gives a very good control of how many items can be in memory at the same time, making your process memory allocation very predictable, and even executions more efficient, the goal of this fix was to NOT change this behaviour, therefore, fixing this bug took more effort than simply changing the back-pressure to be per execution.
The fix consisted of generating a runId for each execution, so this ID allows to control the back-pressure isolated, and reset all the "pending items" for a given run whenever the flow for the given execution is ended, no matter if it had an error or not.
In the fix, the runId was intentionally not part of the ValueBag, this was done to avoid internal values of Caminho being available to the step functions, therefore, the OperatorApplierWithRunId was created.