bash 维护 cronjob 和 shell 脚本的最佳实践？

Best practices for maintaining cronjobs and shell scripts?

提问by Azazo

I have inherited a sprawling crontab that I need to maintain and update. I don't have much experience with it or bash scripting (I think I've got a decent grip on the basics) and I want to do a good job. Short request: Any guidelines for 'refactoring' a messy crontab and set of bash scripts

我继承了需要维护和更新的庞大 crontab。我对它或 bash 脚本编写没有太多经验（我认为我已经掌握了基础知识），我想做得很好。简短请求：“重构”凌乱的 crontab 和一组 bash 脚本的任何指南

Long request: I've run into a number of issues, but are so many people using cron files etc that I feel like I must be missing some large repository of information, best practices and tools - or is this just a stylistic difference for this kind of programming? (My bias: why do something manually if I can use a tool to do it faster, consistently and well?).

长请求：我遇到了许多问题，但是有太多人使用 cron 文件等，我觉得我必须缺少一些大型信息库、最佳实践和工具 - 或者这只是风格上的差异什么样的编程？（我的偏见：如果我可以使用工具更快、更一致、更好地完成某件事，为什么还要手动做某事？）。

Examples of issues so far:

到目前为止的问题示例：

1. Due to an external event, the crontab didn't run for a couple of days. Along with someone else, we manually went through the list, trying to figure out what didn't run, what we needed to rerun, and what scripts we needed to edit and run with earlier dates etc. What I can't find:

   • There are plenty of (slightly pointless) 'cron generators' online. Where are the reverse? Something I can feed in a long crontab, two dates, and have it output which processes should have run when, or just how many times total? This seems within my meager scripting capabilities, so shouldn't it exist already? ;)
   • Alternatively, if I ever have to do that again, is there some way of calling a bashscript so that any instances of date() are pre-set to an earlier time, rather than changing every date call within the script? (e.g. for all the missed reports and billing invoices)

2. It turns out a particular report hadn't been running for two years. It was just requested again, and lo, there it was in the crontab! The bash script just had broken path references to the relevant files. What I can't find: some kind of path checker for bash files? Like a website link checker. Yes I'll be going through these all manually eventually, but it'd show up some at least some of the problem areas.

3. It sounds like some times, there has either been too long or short a gap between dependent processes, so updates have happened after the first has been run, or the first hasn't finished running before the second has been called. I've seen a few possible options for this (eg anacron runs in sequential order), but what would you recommend?

4. There are also a large number of essentially meaningless emails generated from the crontab (scripts throwing errors but running 'correctly', failing mostly silently, or just printing everystep of non-essential scripts). I'll be manually going through scripts and trying to get them to provide more useful data, or 'succeed quietly', but y'know - any guidelines?

1. 由于外部事件，crontab 有几天没有运行。我们和其他人一起手动浏览了列表，试图找出哪些没有运行，哪些需要重新运行，哪些脚本需要在较早的日期编辑和运行等等。我找不到什么：

   • 网上有很多（有点无意义的）“cron 生成器”。反面在哪里？我可以在一个很长的 crontab 中输入两个日期，并让它输出哪些进程应该在何时运行，或者总共运行多少次？这似乎在我微薄的脚本能力范围内，所以它不应该已经存在了吗？;)
   • 或者，如果我不得不再次这样做，是否有某种方法可以调用 bashscript，以便将 date() 的任何实例预设为更早的时间，而不是更改脚本中的每个日期调用？（例如，对于所有错过的报告和账单发票）

2. 事实证明，一份特定的报告已经两年没有运行了。刚刚再次请求它，瞧，它在 crontab 中！bash 脚本只是破坏了对相关文件的路径引用。我找不到什么：bash 文件的某种路径检查器？就像一个网站链接检查器。是的，我最终会手动完成所有这些，但它至少会显示一些问题区域。

3. 听起来有时，依赖进程之间的间隔太长或太短，因此在第一个运行后发生更新，或者在调用第二个之前第一个尚未完成运行。我已经看到了一些可能的选项（例如 anacron 按顺序运行），但是您会推荐什么？

4. 还有大量从 crontab 生成的本质上毫无意义的电子邮件（脚本抛出错误但“正确”运行，大多数情况下无声无息地失败，或者只是打印非必要脚本的每一步）。我将手动检查脚本并试图让它们提供更多有用的数据，或者“悄悄地成功”，但你知道 - 有什么指导方针吗？

If my understanding or layout of the issue is confused, then I apologize, but hey - you see my problem then! I need to go from newbie, to knowing what to do to get this right, and not screw up a touchy system further. Thanks!

如果我对问题的理解或布局感到困惑，那么我深表歉意，但是嘿 - 你看到我的问题了！我需要去从新手，知道怎样做才能得到这个正确的，并没有进一步搞砸了一个敏感的系统。谢谢！

回答by Azazo

Not a full answer, but more resources that have been helpful: http://blog.endpoint.com/2008/12/best-practices-for-cron.html

不是完整的答案，而是更多有用的资源：http: //blog.endpoint.com/2008/12/best-practices-for-cron.html

I am slowly going through this, and trying to implement each of the points. I hadn't thought to google 'best practices cron' til after my post. :P

我正在慢慢地经历这个，并试图实现每一个要点。直到我的帖子之后，我才想到谷歌“最佳实践 cron”。:P

For version control, I'm just going to use RCS in the meantime, as I edit scripts on a file-by-file basis, but I've been advised to get Git set up (or Mercurial if I was on a Windows system).

对于版本控制，我将同时使用 RCS，因为我在逐个文件的基础上编辑脚本，但有人建议我设置 Git（如果我在 Windows 系统上，则设置为 Mercurial ）。

This actually sounds great: http://everythingsysadmin.com/2010/09/xed-202-released.html"xed is a perl script that locks a file, runs $EDITOR on the file, then unlocks it."...and puts it in RCS if it wasn't already. Completely brainless version control. If I get my head around bash, I'd like to create an editing shortcut that automatically commits to whichever version control system I use.

这实际上听起来很棒：http: //everythingsysadmin.com/2010/09/xed-202-released.html“xed 是一个 perl 脚本，可以锁定文件，在文件上运行 $EDITOR，然后解锁它。”...如果还没有，则将其放入 RCS。完全无脑的版本控制。如果我对 bash 有所了解，我想创建一个编辑快捷方式，该快捷方式可以自动提交到我使用的任何版本控制系统。

Other tips I received from an System Admin, Dates: Rather than using say, date, or --date="last monday", use a fixed date and add a day/week etc to it each time it runs (if not more than current day obviously), because then if the script doesn't run, I can just re-run the script repeatedly until it catches up. Ah! (And, this might sound obvious, but heaps of the reports I'll be eventually edit, don't say prominentlywhat dates the report is running for. Will fix.)

我从系统管理员那里收到的其他提示，日期：与其使用日期或 --date="last monday"，不如使用固定日期并在每次运行时为其添加一天/一周等（如果不超过显然是当天），因为如果脚本不运行，我可以重复重新运行脚本，直到它赶上。啊! （而且，这听起来可能很明显，但是我最终将编辑的大量报告，不要突出说明报告的运行日期。将修复。）

And was reassured I should try and get the cron emails as quiet as possible, so that I actually notice if there's an error email. There are wrappers for better cron error reporting that I have not yet investigated, linked here: http://habilis.net/cronic/

并且让我放心，我应该尝试让 cron 电子邮件尽可能安静，这样我才能真正注意到是否有错误电子邮件。有更好的 cron 错误报告的包装器，我还没有研究过，链接在这里：http: //habilis.net/cronic/

回答by sarnold

Herculean task ahead of you, best of luck. :)

你面前的艰巨任务，祝你好运。:)

I'd suggest finding all the tasks that run daily and shove them into their own scripts in /etc/cron.daily/. Same for weekly into /etc/cron.weekly, hourly, and monthly.

我建议找到每天运行的所有任务，并将它们放入/etc/cron.daily/. 每周/etc/cron.weekly、每小时和每月都相同。

You might want to investigate use of anacron(8)for scheduling your jobs, if the machine won't always be online, but you still need some level of control over when the jobs are run. It's been the default cron-helper-tool for multiple distributions for a few years, so hopefully it's stable enough to rely on for your own tasks; but I could easily imagine that it might not perfectly meet your needs.

anacron(8)如果机器不会始终在线，您可能想要调查用于安排作业的用途，但您仍然需要对作业何时运行进行某种程度的控制。几年来，它一直是多个发行版的默认 cron-helper-tool，所以希望它足够稳定，可以依赖于您自己的任务；但我可以很容易地想象它可能无法完全满足您的需求。

Faking the dates to scripts can be done with at least two packages on Ubuntu: datefudgeand faketime. I have no experience with either, but both sound like they should be able to help. I hope you won't need it in the future. :)

可以在 Ubuntu 上使用至少两个包来伪造脚本的日期：datefudge和faketime. 我对两者都没有经验，但听起来他们都应该能够提供帮助。我希望你将来不需要它。:)

Sorry, I know of no path-checker for bash scripts. It seems unlikely, since simple scripts are simple and easy to check by eye :) and complex scripts will be generating their pathnames at runtime anyhow. Maybe you could keep a database of pathnames used by each script and write a new script to verify that database regularly.

抱歉，我不知道 bash 脚本的路径检查器。这似乎不太可能，因为简单的脚本很简单，而且很容易用眼睛检查:) 无论如何，复杂的脚本将在运行时生成它们的路径名。也许您可以保留每个脚本使用的路径名的数据库，并编写一个新脚本来定期验证该数据库。

You coulddisable the cron email by setting MAILTO="". I'm not sure I like this. Maybe setting MAILTOto a logging-only account would help the deluge. Another option is getting really good at your procmail(1)rules so you can stuff them in another mailbox completely.

您可以通过设置禁用 cron 电子邮件MAILTO=""。我不确定我喜欢这个。也许设置MAILTO为仅限日志记录的帐户会有助于洪水泛滥。另一种选择是非常擅长你的procmail(1)规则，这样你就可以把它们完全塞进另一个邮箱。

Getting good at muttcoloror scorecontrols can help you spot the wheat amongst the chaff. (color index red black ERRORor similar commands might help you spot the problems more quickly.)

擅长muttcolor或score控制可以帮助您在谷壳中发现小麦。（color index red black ERROR或类似的命令可能会帮助您更快地发现问题。）