As the included diagram portrays, in a common workflow, the worker instance will consume messages sent to a specified Amazon SQS from another service (e.g.: a web server or another worker). These messages will be received by the worker via POST requests. This eliminates the necessity of configuring a worker as an always-on service, as well as having to add code for reading and consuming messages from an AWS SQS queue. In other words, the worker is implemented as a standard RESTful API/Service that will react to a message sent to it at an specific endpoint via a POST request. This is an awesome approach by Amazon to microservices and reactive design.

The conversion of the SQS message to a POST request is executed by what AWS calls the "SQS Daemon" or "Sqsd". This is a simple daemon they pre-install in the worker tier instances that is constantly monitoring an specific AWS SQS queue (provided by configuration) for new messages. When new messages arrive, it constructs a POST request and sends it to a specific endpoint (also provided via configuration). If the endpoint consumes it without errors and returns a 2** HTTP Code in its response, the "Sqsd" deletes the message from the queue to signal that its consumption was successful.

However, even though this approach is extremely powerful, Amazon does not provide the code of this daemon as open source. Therefore, we have reproduced its behavior by creating our own version of the "Sqsd" free for everyone to use. Moreover, we have provided lots of customization and configuration properties so that it can be molded to your specific use cases.

Elastic Beanstalk simplifies this process by managing the Amazon SQS queue and running a daemon process on each instance that reads from the queue for you. When the daemon pulls an item from the queue, it sends an HTTP POST request locally to http://localhost/ with the contents of the queue message in the body. All that your application needs to do is perform the long-running task in response to the POST. You can configure the daemon to post to a different path, use a MIME type other than application/JSON, connect to an existing queue, or customize connections, timeouts, and retries.

Depuis hier le 22 Février 2017

seems cool and powerful

Default listener (80) is enabled by default, to disable it :
aws:elb:listener:
ListenerEnabled: false

On peut avoir besoin de faire tourner un ou plusieurs cron dans un environnement Beanstalk composé de X instances
Etant donné que toute les instances sont configurées de la meme manière, si on rajoute le cron dans la config, il sera éxecuté sur toutes les instances

Une solution serait de determiner si on est le "runnner" ou non avant de lancer le cron

Pour que toutes les instances sachent qui est le runner, on peut imaginer un système comme celui ci :

• on interroge l'ELB avec les API pour avoir ses backends
• on trie cette liste de backend
• le runner est le premier de cette liste
• si notre ip est égale à celle du runner, on lance le cron

ça a l'avantage de fonctionner quel que soit le nombre d'instances et ne pas stocker un état (on utilise les infos de l'ELB)

Et si on ne veut pas faire tourner tous les crons sur le runner, on pourrait déterminer runnner1 runnner2 runner3 etc suivant l'ordre et faire tourner certains crons si on est runner1 ou bien runner2