windows 使用 cURL/libcurl 上传到 Amazon S3
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/1496453/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Uploading to Amazon S3 using cURL/libcurl
提问by Zaid Amir
I am currently trying to develop an application to upload files to an Amazon S3 bucket using cURL and c++. After carefully reading the S3 developers guide I have started implementing my application using cURL and forming the Header as described by the Developers guide and after lots of trials and errors to determine the best way to create the S3 signature, I am now facing a 501 error. The received header suggests that the method I'm using is not implemented. I am not sure where I'm wrong but here is the HTTP header that I'm sending to amazon:
我目前正在尝试开发一个应用程序，使用 cURL 和 C++ 将文件上传到 Amazon S3 存储桶。在仔细阅读 S3 开发人员指南后，我开始使用 cURL 实现我的应用程序并按照开发人员指南的描述形成标题，经过大量试验和错误以确定创建 S3 签名的最佳方式后，我现在面临 501 错误. 收到的标头表明我正在使用的方法没有实现。我不确定我错在哪里，但这是我发送给亚马逊的 HTTP 标头：
PUT /test1.txt HTTP/1.1
Accept: */*
Transfer-Encoding: chunked
Content-Type: text/plain
Content-Length: 29
Host: [BucketName].s3.amazonaws.com
Date: [Date]
Authorization: AWS [Access Key ID]:[Signature]
Expect: 100-continue
I have truncated the Bucket Name, Access Key ID and Signature for security reasons.
出于安全原因，我截断了存储桶名称、访问密钥 ID 和签名。
I am not sure what I'm doing wrong but I think that the error is generating because of the Accept and Transfer-Encoding Fields (Not Really Sure). So can anyone tell me what I'm doing wrong or why I'm getting a 501.
我不确定我做错了什么，但我认为错误是由于 Accept 和 Transfer-Encoding 字段（不确定）而产生的。所以谁能告诉我我做错了什么或者为什么我得到了 501。
采纳答案by Zaid Amir
Solved: was missing an CURLOPT for the file size in my code and now everything is working perfectly
已解决：在我的代码中缺少文件大小的 CURLOPT，现在一切正常
回答by james
You could execute a bash file. Here is an example upload.shscript which you could just run as: sh upload.sh yourfile
你可以执行一个 bash 文件。这是一个示例upload.sh脚本，您可以将其运行为：sh upload.sh yourfile
#!/bin/bash
file=
bucket=YOUR_BUCKET
resource="/${bucket}/${file}"
contentType="application/x-itunes-ipa"
dateValue=`date -R`
stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
s3Key=YOUR_KEY_HERE
s3Secret=YOUR_SECRET
echo "SENDING TO S3"
signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`
curl -vv -X PUT -T "${file}" \
-H "Host: ${bucket}.s3.amazonaws.com" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: AWS ${s3Key}:${signature}" \
https://${bucket}.s3.amazonaws.com/${file}
more on: http://www.jamesransom.net/?p=58
更多信息：http: //www.jamesransom.net/?p=58
回答by berhauz
The game changed significantly since the question was asked, the simple authorization headers no longer apply, yet it is still feasible to perform with a UNIX shell script, as follows.
自从提出问题以来，游戏发生了重大变化，简单的授权标头不再适用，但仍然可以使用 UNIX shell 脚本执行，如下所示。
Ensure 'openssl' and 'curl' are available at the command line.
确保“openssl”和“curl”在命令行中可用。
Beware, a single extra newline or space character, else the use of CRLF in place of the NewLine char alone would defeat the signature. Note too that you may want to use content types possibly with encodings to prevent any data transformation through the communication media. You may then have to adjust the list of signed headers at several places; please refer to AMAZON S3 API docsfor the numerous conventions to keep enforced like alphabetical-lowercase ordering of header info used in hash calculations at several (redundant) places.
请注意，单个额外的换行符或空格字符，否则单独使用 CRLF 代替 NewLine 字符会破坏签名。另请注意，您可能希望将内容类型与编码一起使用，以防止通过通信媒体进行任何数据转换。然后，您可能需要在多个位置调整签名标题列表；请参阅AMAZON S3 API 文档，以了解在多个（冗余）位置的哈希计算中使用的标头信息的字母小写排序等众多约定。
# BERHAUZ Nov 2019 - curl script for file upload to Amazon S3 Buckets
test -n "" || {
echo "usage: ##代码## <myFileToSend.txt>"
echo "... missing argument file ..."
exit
}
yyyymmdd=`date +%Y%m%d`
isoDate=`date --utc +%Y%m%dT%H%M%SZ`
# EDIT the next 4 variables to match your account
s3Bucket="myBucket.name.here"
bucketLocation="eu-central-1"
s3AccessKey="THISISMYACCESSKEY123"
s3SecretKey="ThisIsMySecretKeyABCD1234efgh5678"
#endpoint="${s3Bucket}.s3-${bucketLocation}.amazonaws.com"
endpoint="s3-${bucketLocation}.amazonaws.com"
fileName=""
contentLength=`cat ${fileName} | wc -c`
contentHash=`openssl sha -sha256 -hex ${fileName} | sed 's/.* //'`
canonicalRequest="PUT\n/${s3Bucket}/${fileName}\n\ncontent-length:${contentLength}\nhost:${endpoint}\nx-amz-content-sha256:${contentHash}\nx-amz-date:${isoDate}\n\ncontent-length;host;x-amz-content-sha256;x-amz-date\n${contentHash}"
canonicalRequestHash=`echo -en ${canonicalRequest} | openssl sha -sha256 -hex | sed 's/.* //'`
stringToSign="AWS4-HMAC-SHA256\n${isoDate}\n${yyyymmdd}/${bucketLocation}/s3/aws4_request\n${canonicalRequestHash}"
echo "----------------- canonicalRequest --------------------"
echo -e ${canonicalRequest}
echo "----------------- stringToSign --------------------"
echo -e ${stringToSign}
echo "-------------------------------------------------------"
# calculate the signing key
DateKey=`echo -n "${yyyymmdd}" | openssl sha -sha256 -hex -hmac "AWS4${s3SecretKey}" | sed 's/.* //'`
DateRegionKey=`echo -n "${bucketLocation}" | openssl sha -sha256 -hex -mac HMAC -macopt hexkey:${DateKey} | sed 's/.* //'`
DateRegionServiceKey=`echo -n "s3" | openssl sha -sha256 -hex -mac HMAC -macopt hexkey:${DateRegionKey} | sed 's/.* //'`
SigningKey=`echo -n "aws4_request" | openssl sha -sha256 -hex -mac HMAC -macopt hexkey:${DateRegionServiceKey} | sed 's/.* //'`
# then, once more a HMAC for the signature
signature=`echo -en ${stringToSign} | openssl sha -sha256 -hex -mac HMAC -macopt hexkey:${SigningKey} | sed 's/.* //'`
authoriz="Authorization: AWS4-HMAC-SHA256 Credential=${s3AccessKey}/${yyyymmdd}/${bucketLocation}/s3/aws4_request, SignedHeaders=content-length;host;x-amz-content-sha256;x-amz-date, Signature=${signature}"
curl -v -X PUT -T "${fileName}" \
-H "Host: ${endpoint}" \
-H "Content-Length: ${contentLength}" \
-H "x-amz-date: ${isoDate}" \
-H "x-amz-content-sha256: ${contentHash}" \
-H "${authoriz}" \
http://${endpoint}/${s3Bucket}/${fileName}
I must acknowledge that, for someone a bit involved in cryptography like me, the Amazon signature scheme deserves numerous critics:
我必须承认，对于像我这样有点涉足密码学的人来说，亚马逊签名方案值得无数批评：
- there's much redundancy in the information being signed,
- the 5 step HMAC cascade is almost inverting semantics between key seed and data where 1 step would suffice with proper usage and same security
- the last 12 characters of the secret key are useless here, because the significant key length of a SHA256 HMAC is ... 256 bits, hence 32 bytes, of which the first 4 always start with "AWS4" for just no purpose.
- overall AWS S3 API re-invents standards where a S/MIME payload would have done
- 签署的信息有很多冗余，
- 5 步 HMAC 级联几乎颠倒了密钥种子和数据之间的语义，其中 1 步就足够了正确使用和相同的安全性
- 秘密密钥的最后 12 个字符在这里没有用，因为 SHA256 HMAC 的重要密钥长度是 ... 256 位，因此是 32 个字节，其中前 4 个总是以“AWS4”开头，只是毫无意义。
- 整体 AWS S3 API 重新发明了 S/MIME 有效负载本应完成的标准
Apologize for the critics, I was not able to resist. Yet acknowledge: it is working reliably, useful for many companies, and an interesting service with a rich API.
为批评者道歉，我无法抗拒。然而承认：它运行可靠，对许多公司有用，并且是具有丰富 API 的有趣服务。
