As mentioned in my previous blog AWS TIPS AND TRICKS: TIME ZONE I have had to do a few things I didn’t know how to do when setting up an EC2 Linux server and I thought I would share these learnings.
How to automatically create a cron job at Instance creation.
I needed to create a schedule to run a small python script every 5 min, the example I was using suggested:
echo “*/5 * * * * python my-python-script” >> /etc/crontab
I included this line in my user data section of the instance however when the instance started the script wasn’t running.
Taking a step back and I tried running the line of code manually, and no matter what I did (adding sudo etc) I always ended up with the same error:
/etc/crontab: Permission denied
Taking another step back, I asked:
Could I display/list the crontab for my user?
no crontab for ec2-user
Could I edit the crontab for my user?
Added the line:
*/5 * * * * python my-python-script
And saved, noted it returned:
crontab: installing new crontab
At this point the script was running, however, this was set-up manually how could I do automatically
One final step back, during my “googling” of my issues I noted there was mention of another way of using cron to schedule and it was by placing a file containing your cron code/schedule (with the addition of which user to run as) in the folder:
From what I can tell that this is a centralised way of maintaining cron schedules especially if you have multiple admin users rather than each admin adding to their own user crontab; they could add their schedule files in the cron.d folder which makes it easier for the other admins to find etc (I could be totally wrong with this understanding so let me know if I am).
As I was already uploading some files from s3 when the instance was created (via s3 sync command – another blog to be written) I saw using the cron.d folder as a potential option.
I created a file called ec2userschedule (note no use of special characters or even a file extension; it didn’t like it when it did) containing the schedule line with the addition of ec2-user
*/5 * * * * ec2-user python my-python-script
I added this file to my s3 bucket that contained the other files I was uploading.
The final thing I had to do was add a file move command to the instance user data after my s3 sync:
mv /home/ec2-user/ec2userschedule /etc/cron.d/
And to my surprise, my python script was run every 5 minutes, yay.
Cron job set at instance start-up. Job done.
If you are looking there is plenty of information in the land of google on how to use cron (all be it on different Linux flavours), I found what I needed by searching ‘cron syntax’.
Barry, Preventer of Chaos
Barry blogs about how to stop chaos in your systems.
You can read all of Barry’s blogs here.
Don’t forget, we can train your team in the art of agile business intelligence at any time!