AWS S3:如何使用 bash 检查存储桶中是否存在文件

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/41871948/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-09-08 22:30:02  来源:igfitidea点击:

AWS S3: How to check if a file exists in a bucket using bash

bashamazon-web-servicesamazon-s3

提问by J. Swaelen

I'd like to know if it's possible to check if there are certain files in a certain bucket.

我想知道是否可以检查某个存储桶中是否有某些文件。

This is what I've found:

这是我发现的:

Checking if a file is in a S3 bucket using the s3cmd

使用 s3cmd 检查文件是否在 S3 存储桶中

It should fix my problem, but for some reason it keeps returning that the file doesn't exist, while it does. This solution is also a little dated and doesn't use the doesObjectExistmethod.

它应该可以解决我的问题,但由于某种原因,它不断返回该文件不存在,而它确实存在。这个解决方案也有点过时,不使用该doesObjectExist方法。

Summary of all the methods that can be used in the Amazon S3 web service

可以在 Amazon S3 Web 服务中使用的所有方法的摘要

This gives the syntax of how to use this method, but I can't seem to make it work.

这给出了如何使用此方法的语法,但我似乎无法使其工作。

Do they expect you to make a boolean variable to save the status of the method, ordoes the function directly give you an output / throw an error?

他们是希望你创建一个布尔变量来保存方法的状态,还是函数直接给你一个输出/抛出一个错误?

This is the code I'm currently using in my bash script:

这是我目前在 bash 脚本中使用的代码:

existBool=doesObjectExist(${BucketName}, backup_${DomainName}_${CurrentDate}.zip)

if $existBool ; then
        echo 'No worries, the file exists.'
fi

I tested it using only the name of the file, instead of giving the full path. But since the error I'm getting is a syntax error, I'm probably just using it wrong.

我仅使用文件名对其进行了测试,而不是提供完整路径。但由于我得到的错误是语法错误,我可能只是使用错误。

Hopefully someone can help me out and tell me what I'm doing wrong.

希望有人可以帮助我并告诉我我做错了什么。

!Edit

!编辑

I ended up looking for another way to do this since using doesObjectExistisn't the fastest or easiest.

我最终寻找另一种方法来做到这一点,因为使用doesObjectExist不是最快或最简单的。

采纳答案by Dave Maple

Last time I saw performance comparisons getObjectMetadatawas the fastest way to check if an object exists. Using the AWS cli that would be the head-objectmethod, example:

上次我看到性能比较getObjectMetadata是检查对象是否存在的最快方法。使用 AWS cli 作为head-object方法,例如:

aws s3api head-object --bucket www.codeengine.com --key index.html

which returns:

返回:

{
    "AcceptRanges": "bytes",
    "ContentType": "text/html; charset=utf-8",
    "LastModified": "Sun, 08 Jan 2017 22:49:19 GMT",
    "ContentLength": 38106,
    "ContentEncoding": "gzip",
    "ETag": "\"bda80810592763dcaa8627d44c2bf8bb\"",
    "StorageClass": "REDUCED_REDUNDANCY",
    "CacheControl": "no-cache, no-store",
    "Metadata": {}
}

回答by Michael Glenn

Note that "aws s3 ls" does not quite work, even though the answer was accepted. It searches by prefix, not by a specific object key. I found this out the hard way when someone renamed a file by adding a '1' to the end of the filename, and the existence check would still return True.

请注意,即使答案被接受,“aws s3 ls”也不太有效。它按前缀搜索,而不是按特定对象键搜索。当有人通过在文件名末尾添加“1”来重命名文件时,我发现这一点很困难,并且存在性检查仍会返回 True。

(Tried to add this as a comment, but do not have enough rep yet.)

(尝试将此添加为评论,但还没有足够的代表。)

回答by ItayB

Following to @DaveMaple & @MichaelGlenn answers, here is the condition I'm using:

按照@DaveMaple & @MichaelGlenn 的回答,这是我使用的条件:

aws s3api head-object --bucket <some_bucket> --key <some_key> || not_exist=true
if [ $not_exist ]; then
  echo "it does not exist"
else
  echo "it exists"
fi

回答by traceformula

One simple way is using aws s3 ls

一种简单的方法是使用 aws s3 ls

exists=$(aws s3 ls $path_to_file)
if [ -z "$exists" ]; then
  echo "it does not exist"
else
  echo "it exists"
fi

回答by Amri

I usually use set -eufo pipefailand the following works better for me because I do not need to worry about unset variables or the entire script exiting.

我通常使用set -eufo pipefail并且以下对我来说效果更好,因为我不需要担心未设置的变量或整个脚本退出。

object_exists=$(aws s3api head-object --bucket $bucket --key $key || true)
if [ -z "$object_exists" ]; then
  echo "it does not exist"
else
  echo "it exists"
fi