Emitting UserData Events With Bosky
This article assumes intermediate knowledge of an EC2 instance’s lifecycle and various AWS services. When an Linux EC2 instance starts up, user data runs as part of the cloud-init system. This allows system administrators to configure an EC2 instance at runtime, exactly once, as user data does not automatically run ever again once the server has started. At Unbounce, we built our user data to install our services onto the machine at runtime, then configure it for the specific environment (production, staging, etc.) that it required. One issue was what to do with failures within the user data script. When a failure happens and fails to start the service successfully, the machine (if behind an auto-scaling group) will be terminated and we lose why the user data failed. The first fix for this is to use off-box logging, like Cloudwatch Logs or, in our case, SumoLogic. That helped, but the turnaround time between the log service receiving the log entry and developers being notified can reach upwards of 5 minutes. By that time, the box is dead and gone. And this all assumes that enough user data ran to configure and start the off-site logging service successfully. ...