实时操作系统 (RTOS) 上的 Python

Python on an Real-Time Operation System (RTOS)

提问by G?khan Sever

I am planning to implement a small-scale data acquisition system on an RTOS platform. (Either on a QNX or an RT-Linux system.)

我计划在 RTOS 平台上实现一个小规模的数据采集系统。（在 QNX 或 RT-Linux 系统上。）

As far as I know, these jobs are performed using C / C++ to get the most out of the system. However I am curious to know and want to learn some experienced people's opinions before I blindly jump into the coding action whether it would be feasible and wiser to write everything in Python (from low-level instrument interfacing through a shiny graphical user interface). If not, mixing with timing-critical parts of the design with "C", or writing everything in C and not even putting a line of Python code.

据我所知，这些工作是使用 C/C++ 执行的，以充分利用系统。但是，我很想知道并想了解一些有经验的人的意见，然后再盲目地投入到编码行动中，用 Python 编写所有内容是否可行和更明智（从低级仪器接口到闪亮的图形用户界面）。如果没有，将设计的时序关键部分与“C”混合，或者用 C 编写所有内容，甚至不放置一行 Python 代码。

Or at least wrapping the C code using Python to provide an easier access to the system.

或者至少使用 Python 包装 C 代码以提供对系统的更轻松访问。

Which way would you advise me to work on? I would be glad if you point some similar design cases and further readings as well.

你会建议我以哪种方式工作？如果您指出一些类似的设计案例和进一步阅读，我会很高兴。

Thank you

谢谢

NOTE1:The reason of emphasizing on QNX is due to we already have a QNX 4.25 based data acquisition system (M300) for our atmospheric measurement experiments. This is a proprietary system and we can't access the internals of it. Looking further on QNX might be advantageous to us since 6.4 has a free academic licensing option, comes with Python 2.5, and a recent GCC version. I have never tested a RT-Linux system, don't know how comparable it to QNX in terms of stability and efficiency, but I know that all the members of Python habitat and non-Python tools (like Google Earth) that the new system could be developed on works most of the time out-of-the-box.

NOTE1：之所以强调QNX，是因为我们已经有一个基于QNX 4.25的数据采集系统（M300）用于我们的大气测量实验。这是一个专有系统，我们无法访问它的内部结构。进一步研究 QNX 可能对我们有利，因为 6.4 具有免费的学术许可选项，附带 Python 2.5 和最新的 GCC 版本。我从来没有测试过 RT-Linux 系统，不知道它在稳定性和效率方面与 QNX 相比如何，但我知道 Python 栖息地和非 Python 工具（如谷歌地球）的所有成员都认为新系统大多数时候可以在开箱即用的作品上进行开发。

采纳答案by Mark Rushakoff

I can't speak for everydata acquisition setup out there, but mostof them spend most of their "real-time operations" waitingfor data to come in -- at least the ones I've worked on.

我不能说那里的每一个数据采集​​设置，但他们中的大多数人将大部分“实时操作”花费在等待数据进来——至少是我工作过的那些。

Then when the data doescome in, you need to immediately record the event or respond to it, and then it's back to the waiting game. That's typically the most time-critical part of a data acquisition system. For that reason, I would generallysay stick with C for the I/O parts of the data acquisition, but there aren't any particularly compelling reasons not to use Python on the non-time-critical portions.

然后当数据确实进来时，你需要立即记录事件或响应它，然后回到等待游戏。这通常是数据采集系统中时间最关键的部分。出于这个原因，我通常会说在数据采集的 I/O 部分坚持使用 C，但没有任何特别令人信服的理由不在非时间关键部分不使用 Python。

If you have fairly loose requirements -- only needs millisecond precision, perhaps -- that adds some more weight to doing everything in Python. As far as development time goes, if you're already comfortable with Python, you would probably have a finished product significantly sooner if you were to do everything in Python and refactor only as bottlenecks appear. Doing the bulk of your work in Python will also make it easier to thoroughly test your code, and as a general rule of thumb, there will be fewer lines of code and thus less room for bugs.

如果您有相当宽松的要求——也许只需要毫秒精度——这会增加在 Python 中做所有事情的权重。就开发时间而言，如果您已经对 Python 感到满意，那么如果您在 Python 中完成所有工作并且仅在出现瓶颈时进行重构，那么您可能会更快地获得成品。使用 Python 完成大部分工作还可以更轻松地彻底测试您的代码，并且作为一般经验法则，代码行会更少，因此出现错误的空间也会更小。

If you need to specifically multi-task(not multi-thread), Stackless Pythonmight be beneficial as well. It's likemulti-threading, but the threads (or tasklets, in Stackless lingo) are not OS-level threads, but Python/application-level, so the overhead of switching between tasklets is significantlyreduced. You can configure Stackless to multitask cooperatively or preemptively. The biggest downside is that blocking IO will generally block your entire set of tasklets. Anyway, considering that QNX is already a real-time system, it's hard to speculate whether Stackless would be worth using.

如果您需要专门处理多任务（而不是多线程），Stackless Python也可能是有益的。这就像多线程，但线程（或用 Stackless 行话来说是 tasklets）不是 OS 级别的线程，而是 Python/应用程序级别，因此 tasklet 之间切换的开销显着减少。您可以将 Stackless 配置为协作或抢占式多任务。最大的缺点是阻塞 IO 通常会阻塞你的整个 tasklet 集。无论如何，考虑到 QNX 已经是一个实时系统，很难推测 Stackless 是否值得使用。

My vote would be to take the as-much-Python-as-possible route -- I see it as low cost and high benefit. If and when you do need to rewrite in C, you'll already have working code to start from.

我的投票将采取尽可能多的 Python 路线——我认为它是低成本和高收益的。如果您确实需要用 C 重写，那么您已经有了可以开始工作的代码。

回答by BobC

I've built several all-Python soft real-time (RT) systems, with primary cycle times from 1 ms to 1 second. There are some basic strategies and tactics I've learned along the way:

我已经构建了几个全 Python 软实时 (RT) 系统，主循环时间从 1 毫秒到 1 秒。在此过程中，我学到了一些基本的策略和技巧：

1. Use threading/multiprocessing onlyto offload non-RT work from the primary thread, where queues between threads are acceptable and cooperative threading is possible (no preemptive threads!).

2. Avoid the GIL. Which basically means not only avoiding threading, but also avoiding system calls to the greatest extent possible, especially during time-critical operations, unless they are non-blocking.

3. Use C modules when practical. Things (usually) go faster with C! But mainly if you don't have to write your own: Stay in Python unless there really is no alternative. Optimizing C module performance is a PITA, especially when translating across the Python-C interface becomes the most expensive part of the code.

4. Use Python accelerators to speed up your code. My first RT Python project greatly benefited from Psyco (yeah, I've been doing this a while). One reason I'm staying with Python 2.x today is PyPy: Things alwaysgo faster with LLVM!

5. Don't be afraid to busy-wait when critical timing is needed. Use time.sleep() to 'sneak up' on the desired time, then busy-wait during the last 1-10 ms. I've been able to get repeatable performance with self-timing on the order of 10 microseconds. Be sure your Python task is run at max OS priority.

6. Numpy ROCKS! If you are doing 'live' analytics or tons of statistics, there is NOway to get more work done faster and with less work (less code, fewer bugs) than by using Numpy. Not in any other language I know of, including C/C++. If the majority of your code consists of Numpy calls, you will be very, very fast. I can't wait for the Numpy port to PyPy to be completed!

7. Be aware of how and when Python does garbage collection. Monitor your memory use, and force GC when needed. Be sure to explicitly disable GC during time-critical operations. All of my RT Python systems run continuously, and Python loves to hog memory. Careful coding can eliminate almost all dynamic memory allocation, in which case you can completely disable GC!

8. Try to perform processing in batches to the greatest extent possible. Instead of processing data at the input rate, try to process data at the output rate, which is often much slower. Processing in batches also makes it more convenient to gather higher-level statistics.

9. Did I mention using PyPy? Well, it's worth mentioning twice.

1. 仅使用线程/多处理从主线程卸载非 RT 工作，其中线程之间的队列是可以接受的，并且可以进行协作线程（没有抢占线程！）。

2. 避免使用 GIL。这基本上意味着不仅要避免线程化，还要尽可能避免系统调用，尤其是在时间关键的操作期间，除非它们是非阻塞的。

3. 实用时使用 C 模块。事情（通常）用 C 来得更快！但主要是如果您不必自己编写：除非确实别无选择，否则请留在 Python 中。优化 C 模块性能是一个 PITA，尤其是当跨 Python-C 接口进行翻译成为代码中最昂贵的部分时。

4. 使用 Python 加速器来加速您的代码。我的第一个 RT Python 项目从 Psyco 中受益匪浅（是的，我已经这样做了一段时间）。我今天继续使用 Python 2.x 的一个原因是 PyPy：使用 LLVM总是更快！

5. 当需要关键时间时，不要害怕忙等待。使用 time.sleep() 在所需时间“潜入”，然后在最后 1-10 毫秒期间忙等待。我已经能够通过大约 10 微秒的自定时获得可重复的性能。确保您的 Python 任务以最大操作系统优先级运行。

6. 麻木的岩石！如果您正在进行“实时”分析或大量统计数据，则没有比使用 Numpy 更快、以更少的工作（更少的代码、更少的错误）完成更多工作的方法。不是我知道的任何其他语言，包括 C/C++。如果您的大部分代码由 Numpy 调用组成，那么您将非常非常快。我等不及将 Numpy 移植到 PyPy 完成了！

7. 请注意 Python 进行垃圾收集的方式和时间。监控您的内存使用情况，并在需要时强制执行 GC。确保在时间关键操作期间显式禁用 GC。我所有的 RT Python 系统都在连续运行，而且 Python 喜欢占用内存。仔细编码可以消除几乎所有的动态内存分配，在这种情况下你可以完全禁用GC！

8. 尽量做到批量处理。不要以输入速率处理数据，而是尝试以输出速率处理数据，这通常要慢得多。批量处理还可以更方便地收集更高级别的统计信息。

9. 我提到使用 PyPy 了吗？嗯，值得提两次。

There are many other benefits to using Python for RT development. For example, even if you are fairly certain your target language can't be Python, it can pay huge benefits to develop and debug a prototype in Python, then use it as both a template and test tool for the final system. I had been using Python to create quick prototypes of the "hard parts" of a system for years, and to create quick'n'dirty test GUIs. That's how my first RT Python system came into existence: The prototype (+Psyco) was fast enough, even with the test GUI running!

使用 Python 进行 RT 开发还有许多其他好处。例如，即使您相当确定您的目标语言不能是 Python，但在 Python 中开发和调试原型，然后将其用作最终系统的模板和测试工具，它可以带来巨大的好处。多年来，我一直在使用 Python 来创建系统“硬部件”的快速原型，并创建快速不脏的测试 GUI。我的第一个 RT Python 系统就是这样诞生的：原型 (+Psyco) 足够快，即使测试 GUI 正在运行！

-BobC

-鲍勃

Edit: Forgot to mention the cross-platform benefits: My code runs pretty much everywhere with a) no recompilation (or compiler dependencies, or need for cross-compilers), and b) almost no platform-dependent code (mainly for misc stuff like file handling and serial I/O). I can develop on Win7-x86 and deploy on Linux-ARM (or any other POSIX OS, all of which have Python these days). PyPy is primarily x86 for now, but the ARM port is evolving at an incredible pace.

编辑：忘了提及跨平台的好处：我的代码几乎可以在任何地方运行，a）没有重新编译（或编译器依赖，或需要交叉编译器），b）几乎没有依赖于平台的代码（主要用于其他类似的东西）文件处理和串行 I/O）。我可以在 Win7-x86 上开发并部署在 Linux-ARM（或任何其他 POSIX 操作系统，现在所有这些操作系统都有 Python）。PyPy 目前主要是 x86，但 ARM 端口正在以惊人的速度发展。

回答by hobbs

Generally the reason advanced against using a high-level language in a real-time context is uncertainty-- when you run a routine one time it might take 100us; the next time you run the same routine it might decide to extend a hash table, calling malloc, then malloc asks the kernel for more memory, which could do anything from returning instantly to returning milliseconds later to returning secondslater to erroring, none of which is immediately apparent (or controllable) from the code. Whereas theoretically if you write in C (or even lower) you can prove that your critical paths will "always" (barring meteor strike) run in X time.

一般来说，反对在实时上下文中使用高级语言的原因是不确定性——当你运行一次例程时，它可能需要 100us；下次运行相同的例程时，它可能会决定扩展一个哈希表，调用 malloc，然后 malloc 向内核请求更多内存，这可以做任何事情，从立即返回到毫秒后返回到秒后返回到错误，没有一个从代码中可以立即看出（或可控）。而理论上，如果您用 C（甚至更低）编写，您可以证明您的关键路径将“始终”（除非流星撞击）在 X 时间内运行。

回答by Andrew Walker

Our team have done some work combining multiple languages on QNX and had quite a lot of success with the approach. Using python can have a big impact on productivity, and tools like SWIGand ctypes make it really easy to optimize code and combine features from the different languages.

我们的团队在 QNX 上完成了一些结合多种语言的工作，并在这种方法上取得了很大的成功。使用 Python 可以对生产力产生重大影响，而SWIG和 ctypes等工具可以非常轻松地优化代码并结合来自不同语言的功能。

However, if you're writing anything time critical, it should almost certainly be written in c. Doing this means you avoid the implicit costs of an interpreted langauge like the GIL (Global Interpreter Lock), and contentionon many small memory allocations. Both of these things can have a big impact on how your application performs.

但是，如果您正在编写任何时间紧迫的东西，则几乎肯定应该用 c 编写。这样做意味着您可以避免像 GIL（全局解释器锁）这样的解释语言的隐性成本，以及对许多小内存分配的争用。这两件事都会对您的应用程序的执行方式产生重大影响。

Also python on QNX tends not to be 100% compatible with other distributions (ie/ there are sometimes modules missing).

此外，QNX 上的 python 往往不会与其他发行版 100% 兼容（即/有时缺少模块）。

回答by XPav

One important note: Python for QNX is generally available only for x86.

一个重要说明：Python for QNX 通常仅适用于 x86。

I'm sure you can compile it for ppc and other archs, but that's not going to work out of the box.

我相信你可以为 ppc 和其他拱门编译它，但这不会开箱即用。