为 BigQuery Python CLI 设置 GOOGLE_APPLICATION_CREDENTIALS

Setting GOOGLE_APPLICATION_CREDENTIALS for BigQuery Python CLI

提问by Colin Ricardo

I'm trying to connect to Google BigQuery through the BigQuery API, using Python.

我正在尝试使用 Python 通过 BigQuery API 连接到 Google BigQuery。

I'm following this page here: https://cloud.google.com/bigquery/bigquery-api-quickstart

我在这里关注这个页面：https: //cloud.google.com/bigquery/bigquery-api-quickstart

My code is as follows:

我的代码如下：

import os
import argparse

from apiclient.discovery import build
from apiclient.errors import HttpError
from oauth2client.client import GoogleCredentials

GOOGLE_APPLICATION_CREDENTIALS = './Peepl-cb1dac99bdc0.json'

def main(project_id):
    # Grab the application's default credentials from the environment.
    credentials = GoogleCredentials.get_application_default()
    print(credentials)
    # Construct the service object for interacting with the BigQuery API.
    bigquery_service = build('bigquery', 'v2', credentials=credentials)

    try:
        query_request = bigquery_service.jobs()
        query_data = {
            'query': (
                'SELECT TOP(corpus, 10) as title, '
                'COUNT(*) as unique_words '
                'FROM [publicdata:samples.shakespeare];')
        }

        query_response = query_request.query(
            projectId=project_id,
            body=query_data).execute()

        print('Query Results:')
        for row in query_response['rows']:
            print('\t'.join(field['v'] for field in row['f']))

    except HttpError as err:
        print('Error: {}'.format(err.content))
        raise err


if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument('project_id', help='Your Google Cloud Project ID.')

    args = parser.parse_args()

    main(args.project_id)

However, when I run this code through the terminal, I get the following error:

但是，当我通过终端运行此代码时，出现以下错误：

oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

As you can see in the code, I've tried to set the GOOGLE_APPLICATION_CREDENTIALSas per the link in the error. However, the error persists. Does anyone know what the issue is?

正如您在代码中看到的那样，我尝试GOOGLE_APPLICATION_CREDENTIALS按照错误中的链接进行设置。但是，错误仍然存​​在。有谁知道问题是什么？

Thank you in advance.

先感谢您。

采纳答案by Roee Anuar

First - Thanks for the code - this provided to be very useful. I would also suggest adding setting the environmental variable directly in your code - as not to set it for every environment you work on. you can use the following code:

首先 - 感谢您的代码 - 这非常有用。我还建议直接在您的代码中添加设置环境变量 - 不要为您工作的每个环境设置它。您可以使用以下代码：

import os
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "path_to_your_.json_credential_file"

I found this useful when switching between different projects that require different credentials.

我发现这在需要不同凭据的不同项目之间切换时很有用。

回答by Michael Sheldon

The link provided in the error message, https://developers.google.com/identity/protocols/application-default-credentials, says to set the environment variableto point at the fail that contains the JSON service credentials. It looks like you set a Python variable. Try setting your terminal's environment variable to point at the correct file.

错误消息中提供的链接https://developers.google.com/identity/protocols/application-default-credentials表示将环境变量设置为指向包含 JSON 服务凭据的失败。看起来您设置了一个 Python 变量。尝试将终端的环境变量设置为指向正确的文件。

An alternate would be to explicitly use some other credentials when you aren't running in a GCE container, like oauth2client.client.SignedJwtAssertionCredentialsand point it directly at your client secret so you don't have to indirect through an environment variable.

当您不在 GCE 容器中运行时，另一种方法是显式使用一些其他凭据，例如oauth2client.client.SignedJwtAssertionCredentials并将其直接指向您的客户端机密，这样您就不必通过环境变量进行间接访问。

回答by Wolf Rendall

It's looking for the environment variable in your local UNIX (or other) environment, not a variable in your python script.

它在您的本地 UNIX（或其他）环境中寻找环境变量，而不是您的 Python 脚本中的变量。

You'd set that by opening up your terminal or cygwin and doing one of the following:

您可以通过打开终端或 cygwin 并执行以下操作之一来设置它：

export GOOGLE_APPLICATION_CREDENTIALS='/path/to/your/client_secret.json'

Type that into your terminal to set the variable for just this session

在您的终端中输入它以仅为此会话设置变量

Open up your .bashrc file, in UNIX by typing in nano ~/.bashrc and add this line to it underneath user specific aliases if you see that header:

打开您的 .bashrc 文件，在 UNIX 中输入 nano ~/.bashrc 并将此行添加到用户特定别名下面，如果您看到该标题：

GOOGLE_APPLICATION_CREDENTIALS="/full/path/to/your/client_secret.json"

Then reload it by typing source ~/.bashrc and confirm that it's set by trying echo $GOOGLE_APPLICATION_CREDENTIALS. If it returns the path, you're good.

然后通过键入 source ~/.bashrc 重新加载它并通过尝试确认它已设置echo $GOOGLE_APPLICATION_CREDENTIALS。如果它返回路径，你就很好。

回答by Questions_Unleashed

It's looking for the environment variable. But I was able to solve this problem on Windows platform by using Application Default Credentials.

它正在寻找环境变量。但是我能够通过使用应用程序默认凭据在 Windows 平台上解决这个问题。

Steps that I followed:

我遵循的步骤：

• Installed Google SDK
• Then execute gcloud initsteps to specify my defaults credentials and default project which you can change as and when needed. gcloudexecutable can be loacted in the bindirectory where you chose to install Google SDK.
• After successfully providing the credentials, you can check in at the location C:\Users\"yourusername"\AppData\Roaming\gcloud\legacy_credentials\"youremail". You can find the credentials stored in the JSON format there.

• 安装谷歌 SDK
• 然后执行gcloud init步骤以指定我的默认凭据和默认项目，您可以根据需要进行更改。gcloud可执行文件可以位于bin您选择安装 Google SDK的目录中。
• 成功提供凭据后，您可以在该位置签到C:\Users\"yourusername"\AppData\Roaming\gcloud\legacy_credentials\"youremail"。您可以在那里找到以 JSON 格式存储的凭据。

It helped me to resolve the error.

它帮助我解决了错误。

回答by Roee Anuar

If you would like to use different credential files without setting the environmental variable, you can use the following code:

如果你想在不设置环境变量的情况下使用不同的凭证文件，可以使用以下代码：

from oauth2client import service_account
from apiclient.discovery import build
import json

client_credentials = json.load(open("<path to .json credentials>"))

credentials_token = service_account._JWTAccessCredentials.from_json_keyfile_dict(client_credentials)

bigquery_service = build('bigquery', 'v2', credentials=credentials_token)
query_request = bigquery_service.jobs()
query_data = {
    'query': (
            'SELECT TOP(corpus, 10) as title, '
            'COUNT(*) as unique_words '
            'FROM [publicdata:samples.shakespeare];')
    }

query_response = query_request.query(
           projectId=project_id,
           body=query_data).execute()

print('Query Results:')
for row in query_response['rows']:
    print('\t'.join(field['v'] for field in row['f']))

回答by arthankamal

I'm not sure about BigQuery, but i'm using Google Data Storefor saving. If you've installed gcloud sdkin your mac, you can try running this command

我不确定BigQuery，但我正在使用它Google Data Store来保存。如果你已经安装gcloud sdk在你的 mac 上，你可以尝试运行这个命令

gcloud auth application-default login

回答by Hakk? Eser

GOOGLE_APPLICATION_CREDENTIALS NOT FOUND ERROR SOLUTION FOR C#

GOOGLE_APPLICATION_CREDENTIALS 未找到 C# 的错误解决方案

System.Environment.SetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS",@"C:\apikey.json");
string Pathsave = System.Environment.GetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS");

回答by Tim Swast

Note: oauth2client is deprecated, instead of GoogleCredentials.get_application_default()you can use google.auth.default(). Install the package first with:

注意：oauth2client 已弃用，GoogleCredentials.get_application_default()您可以使用google.auth.default()代替。首先安装软件包：

pip install google-auth

In your specific example, I see you know where the JSON file is located from your code. Instead of default credentials (from environment variables), you can use a service account directlywith the google.oauth2.service_accountmodule.

在您的具体示例中，我看到您从代码中知道 JSON 文件的位置。您可以直接通过google.oauth2.service_account模块使用服务帐户，而不是默认凭据（来自环境变量）。

credentials = google.oauth2.service_account.from_service_account_file(
    './Peepl-cb1dac99bdc0.json',
    scopes=['https://www.googleapis.com/auth/cloud-platform'])

You can use this credentials file just as you are currently doing so by passing them to googleapiclient.discovery.buildor if you are using the google-cloud-bigquerylibrary, pass the credentials to the google.cloud.bigquery.Clientconstructor.

您可以像当前一样使用此凭据文件，方法是将它们传递给googleapiclient.discovery.build，或者如果您正在使用google-cloud-bigquery库，请将凭据传递给google.cloud.bigquery.Client构造函数。