Python 如何在 Windows 上运行 Airflow

How to run Airflow on Windows

提问by Rafael

The usual instructions for running Airflow do not apply on a Windows environment:

运行 Airflow 的通常说明不适用于 Windows 环境：

# airflow needs a home, ~/airflow is the default,
# but you can lay foundation somewhere else if you prefer
# (optional)
export AIRFLOW_HOME=~/airflow

# install from pypi using pip
pip install airflow

# initialize the database
airflow initdb

# start the web server, default port is 8080
airflow webserver -p 8080

The Airflow utility is not available in the command line and I can't find it elsewhere to be manually added. How can Airflow run on Windows?

Airflow 实用程序在命令行中不可用，我无法在其他地方找到它以手动添加。Airflow 如何在 Windows 上运行？

采纳答案by Ayush

You can activate bashin windows and follow the tutorial as is. I was able to get up and running successfully following above.

您可以bash在 Windows中激活并按原样遵循教程。我能够在上面成功启动并运行。

Once you are done installing, edit airflow.cfgto point all your configurations to somewhere in your windows system rather than lxss (ubuntu) since there are bugs around ubuntu not showing files written by windows system.

完成安装后，编辑airflow.cfg以将所有配置指向 Windows 系统中的某个位置，而不是 lxss (ubuntu)，因为 ubuntu 周围存在错误，不显示由 Windows 系统编写的文件。

回答by Rafael

Instead of installing Airflow via pip, download the zip on the Airflow project's GitHub, unzip it and in its folder, run python setup.py installon the command line. ERROR - 'module' object has no attribute 'SIGALRM'errors will happen, but so far this had no impact on Airflow's functions.

不是通过 pip 安装 Airflow，而是在Airflow 项目的 GitHub上下载 zip ，解压缩它并在其文件夹中，python setup.py install在命令行上运行 。ERROR - 'module' object has no attribute 'SIGALRM'错误会发生，但到目前为止这对 Airflow 的功能没有影响。

Using this method, the airflow util will not be available as a command. As a workaround, use the [current folder]\build\scripts-2.7\airflowfile, which is the python script for the airflow util.

使用此方法，airflow util 将无法作为命令使用。作为解决方法，请使用该[current folder]\build\scripts-2.7\airflow文件，该文件是用于 airflow util 的 python 脚本。

Another solution is to append to the System PATH variable a link to a batch file that runs airflow (airflow.bat):

另一种解决方案是将指向运行气流的批处理文件的链接附加到系统路径变量 (airflow.bat)：

python C:\path\to\airflow %*

From this point, the tutorial may be followed normally:

从这点开始，可以正常遵循教程：

airflow init
airflow webserver -p 8080

I have not tested how well or if Airflow's DAGs run on Windows.

我还没有测试 Airflow 的 DAG 在 Windows 上的运行情况。

回答by Samir Seth

Unfortunately, the answer to this seems to be "No" as of Dec 2015 - see https://github.com/airbnb/airflow/issues/709. This is because of the move to gunicorn. gunicorn may get windows support in R18.

不幸的是，截至 2015 年 12 月，这个问题的答案似乎是“否” - 请参阅https://github.com/airbnb/airflow/issues/709。这是因为转向gunicorn。gunicorn 可能会在 R18 中获得 Windows 支持。

回答by Kyle Bridenstine

You can do it using Cygwin. Cygwin is a command line shell that runs on Windows and emulates Linux. So you'll be able to run the commands,

您可以使用Cygwin 来完成。Cygwin 是一个在 Windows 上运行并模拟 Linux 的命令行 shell。所以你将能够运行命令，

# airflow needs a home, ~/airflow is the default,
# but you can lay foundation somewhere else if you prefer
# (optional)
export AIRFLOW_HOME=~/airflow

# install from pypi using pip
pip install apache-airflow

# initialize the database
airflow initdb

# start the web server, default port is 8080
airflow webserver -p 8080

Note 1:If you're running Cygwin on your company supplied computer you may need to run the Cygwin application as an administrator. You can do so with the following tutorial from Microsoft.

注意 1：如果您在公司提供的计算机上运行 Cygwin，您可能需要以管理员身份运行 Cygwin 应用程序。您可以使用Microsoft 的以下教程来完成此操作。

Note 2:If like me you are behind a proxy (at your work or whatever proxy you're behind) you'll need to set two enviornment variables for pip to work on the command line; in this case Cygwin. You can follow this StackOverflow answerfor more details. So I set the following two environment variables on my Windows machine,

注意 2：如果像我一样你在代理后面（在你的工作或你后面的任何代理），你需要设置两个环境变量让 pip 在命令行上工作；在这种情况下，Cygwin。您可以按照此 StackOverflow 答案了解更多详细信息。所以我在我的 Windows 机器上设置了以下两个环境变量，

// Note this first entry has an S in HTTPS and the other entry is just regular HTTP. Don't forget that distinction in the key name and in the url of the value.
HTTPS_PROXY=https://myUsernameGoesHere:myPasswordGoesHere@yourProxyHostNameGoesHere:yourProxyPortNumberGoesHere

HTTP_PROXY=http://myUsernameGoesHere:myPasswordGoesHere@yourProxyHostNameGoesHere:yourProxyPortNumberGoesHere

No Longer Works:Apparently all of the above work was in vain because Airflow won't work on Windows. Please see this StackOverflow post. The above steps will allow you to use Pip though.

不再有效：显然上述所有工作都是徒劳的，因为 Airflow 无法在 Windows 上运行。请参阅此StackOverflow 帖子。上述步骤将允许您使用 Pip。

Alternatively, and I know this may or may not be seen as being run on Windows, you could install a virtual machine client such as Oracle's Virtualboxor VMware's Workstationand then setup whatever Linux version you want such as Ubuntu Desktopand then you can run Linux normally. If you need more detailed steps to do this you can follow this AskUbuntu from the Stack Exchange community answer here.

或者，我知道这可能会或可能不会被视为在 Windows 上运行，您可以安装虚拟机客户端，例如Oracle 的 Virtualbox或VMware 的 Workstation，然后设置您想要的任何 Linux 版本，例如Ubuntu Desktop，然后您可以运行 Linux一般。如果您需要更详细的步骤来执行此操作，您可以在此处遵循 Stack Exchange 社区中的 AskUbuntu 答案。

Alternatively (2), you could create an AWS account, then setup a simple ec2-instance running Linux, then ssh into that ec2-instance, and then run all your commands to your hearts content. AWS offers a free tierso you should be able to do it for free. Plus, AWS is very well documented so it shouldn't be too hard to get a simple Linux server up and running; I estimate a beginner could be done with it in about an hour.

或者 (2)，您可以创建一个 AWS 帐户，然后设置一个简单的运行 Linux 的 ec2-instance，然后通过ssh 进入该 ec2-instance，然后将所有命令运行到您的心内容中。AWS 提供免费套餐，因此您应该可以免费使用。另外，AWS 有很好的文档记录，因此启动和运行一个简单的 Linux 服务器应该不会太难；我估计初学者可以在大约一个小时内完成。

回答by John Humphreys - w00te

Three Basic Options

三个基本选项

I went through a few iterations of this problem and documented them as I went along. The three things I tried were:

我经历了这个问题的几次迭代，并在我进行的过程中记录了它们。我尝试过的三件事是：

1. Install Airflow directly into Windows 10- This attempt failed.
2. Install Airflow into Windows 10 WSL with Ubuntu- This worked great. Note that WSL is Windows Subsystem for Linux, which you can get for free in the Windows store.
3. Install Airflow into Windows 10 via Docker + Centos- This worked great as well.

1. 将 Airflow 直接安装到 Windows 10- 此尝试失败。
2. 使用 Ubuntu 将 Airflow 安装到 Windows 10 WSL- 这很好用。请注意，WSL 是适用于 Linux 的 Windows 子系统，您可以在 Windows 商店中免费获得。
3. 通过 Docker + Centos 将 Airflow 安装到 Windows 10- 这也很有效。

Note that if you want to get it running as a Linux service, it is not possible for option number 2. It is possible for option number 3, but I didn't do it as it requires activating privileged containers in docker (which I wan't aware of when I started). Also, running a service in Docker is kind of against paradigm as each container should be a single process/unit of responsibility anyway.

请注意，如果您想让它作为 Linux 服务运行，则选项 2 是不可能的。选项 3 是可能的，但我没有这样做，因为它需要在 docker 中激活特权容器（我希望不知道我什么时候开始的）。此外，在 Docker 中运行服务有点违反范式，因为无论如何每个容器都应该是一个单一的进程/责任单位。

Detailed Description of #2 - WSL Option

#2 - WSL 选项的详细说明

If you're gong for option 2, the basic steps are:

如果您选择选项 2，基本步骤是：

• Get WSL Ubuntu installed and opened up.
• Verify it comes with python 3.6.5 or so ("python3 -version").
• Assuming it still does, add these packages so that installing PIP will wor.
  • sudo apt-get install software-properties-common
  • sudo apt-add-repository universe
  • sudo apt-get update
• Install pip with:
  • sudo apt-get install python-pip
• Run the following 2 commands to install airflow:
  • export SLUGIFY_USES_TEXT_UNIDECODE=yes
  • pip install apache-airflow
• Open a new terminal (I was surprised, but this seemed to be required).
• Init the airflow DB:
  • airflow initdb

• 安装并打开 WSL Ubuntu。
• 验证它是否带有 python 3.6.5 左右（“python3 -version”）。
• 假设它仍然存在，请添加这些包，以便安装 PIP。
  • sudo apt-get install software-properties-common
  • sudo apt-add-repository 宇宙
  • sudo apt-get 更新
• 使用以下命令安装 pip：
  • 须藤 apt-get 安装 python-pip
• 运行以下 2 个命令来安装气流：
  • export SLUGIFY_USES_TEXT_UNIDECODE=yes
  • pip 安装 apache-airflow
• 打开一个新终端（我很惊讶，但这似乎是必需的）。
• 初始化气流数据库：
  • 气流初始化数据库

After this, you should be good to go! The blog has more detail on many of these steps and rough timelines for how long setting up WSL takes, etc - so if you have a hard time dive in there some more.

在此之后，你应该很高兴去！该博客提供了有关其中许多步骤的更多详细信息，以及设置 WSL 需要多长时间的粗略时间表等 - 因此，如果您在那里遇到困难，请再深入一些。

回答by Rohit

I'm runnig airflow on windows 10 using docker.

我正在使用 docker 在 Windows 10 上运行气流。

1) First you need to install docker on your windows .

1）首先你需要在你的windows上安装docker。

2) Run command docker versionfrom command prompt if you get output means docker installed succesfuuly

2)docker version从命令提示符运行命令，如果你得到输出意味着 docker 安装成功

2) Then you need to pull airflow image using command docker pull puckel/docker-airflow

2）然后你需要使用命令拉气流图像 docker pull puckel/docker-airflow

3) Next step is to run image docker run -d -p 8080:8080 puckel/docker-airflow webserver

3）下一步是运行图像 docker run -d -p 8080:8080 puckel/docker-airflow webserver

4) This will run airflow and you can access webui at localhost:8080

4) 这将运行气流，您可以在 localhost:8080 上访问 webui

5) To copy dags use this command docker cp sample_dag.py containerName:/usr/local/airflow/dags

5) 要复制 dag，请使用此命令 docker cp sample_dag.py containerName:/usr/local/airflow/dags

To access airflow utility you need to access the bash shell of container . you can do so using docker exec -it containerName bash. Once you inside bash shell you can run command line utilities ex **airflow list_dags**

要访问气流实用程序，您需要访问 container 的 bash shell。您可以使用docker exec -it containerName bash. 进入 bash shell 后，您可以运行命令行实用程序，例如**airflow list_dags**

Hope it helps

希望能帮助到你