如何将 S3 存储桶挂载到 EC2 实例并使用 PHP 写入?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/16428552/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-25 11:06:54  来源:igfitidea点击:

How can I mount an S3 bucket to an EC2 instance and write to it with PHP?

phpamazon-s3fuses3fs

提问by Ben Harold

I'm working on a project that is being hosted on Amazon Web Services. The server setup consists of two EC2 instances, one Elastic Load Balancer and an extra Elastic Block Store on which the web application resides. The project is supposedto use S3 for storage of files that users upload. For the sake of this question, I'll call the S3 bucket static.example.com

我正在处理一个托管在 Amazon Web Services 上的项目。服务器设置由两个 EC2 实例、一个 Elastic Load Balancer 和一个额外的 Elastic Block Store 组成,Web 应用程序驻留在其上。该项目应该使用 S3 来存储用户上传的文件。对于这个问题,我将调用 S3 存储桶static.example.com

I have tried using s3fs(https://code.google.com/p/s3fs/wiki/FuseOverAmazon), RioFS(https://github.com/skoobe/riofs) and s3ql(https://code.google.com/p/s3ql/). s3fswill mount the filesystem but won't let me write to the bucket (I asked this question on SO: How can I mount an S3 volume with proper permissions using FUSE). RioFSwill mount the filesystem and will let me write to the bucket from the shell, but files that are saved using PHP don't appear in the bucket (I opened an issue with the project on GitHub). s3qlwill mount the bucket, but none of the files that are already in the bucket appear in the filesystem.

我曾尝试使用s3fshttps://code.google.com/p/s3fs/wiki/FuseOverAmazon)、RioFShttps://github.com/skoobe/riofs)和s3qlhttps://code.google.com/p /s3ql/)。s3fs将挂载文件系统,但不会让我写入存储桶(我在 SO 上问了这个问题:如何使用 FUSE 以适当的权限挂载 S3 卷)。RioFS将挂载文件系统并让我从 shell 写入存储桶,但使用 PHP 保存的文件不会出现在存储桶中(我在 GitHub 上打开了该项目的问题)。s3ql将挂载存储桶,但存储桶中已有的文件都不会出现在文件系统中。

These are the mount commands I used:

这些是我使用的挂载命令:

s3fs static.example.com -ouse_cache=/tmp,allow_other /mnt/static.example.com
riofs -o allow_other http://s3.amazonaws.com static.example.com /mnt/static.example.com
s3ql mount.s3ql s3://static.example.com /mnt/static.example.com

I've also tried using this S3 class: https://github.com/tpyo/amazon-s3-php-class/and this FuelPHP specific S3 package: https://github.com/tomschlick/fuel-s3. I was able to get the FuelPHP package to list the available buckets and files, but saving files to the bucket failed (but did not error).

我也试过使用这个 S3 类:https: //github.com/tpyo/amazon-s3-php-class/和这个 FuelPHP 特定的 S3 包:https: //github.com/tomschlick/fuel-s3。我能够让 FuelPHP 包列出可用的存储桶和文件,但将文件保存到存储桶失败(但没有出错)。

Have you ever mounted an S3 bucket on a local linux filesystem and used PHP to write a file to the bucket successfully? What tool(s) did you use? If you used one of the above mentioned tools, what version did you use?

您是否曾经在本地 linux 文件系统上挂载过 S3 存储桶并成功使用 PHP 将文件写入存储桶?你用的是什么工具?如果您使用上述工具之一,您使用的是哪个版本?

EDITI have been informed that the issue I opened with RioFSon GitHub has been resolved. Although I decided to use the S3 REST APIrather than attempting to mount a bucket as a volume, it seems that RioFSmay be a viable option these days.

编辑我被告知我RioFS在 GitHub 上打开的问题已经解决。尽管我决定使用S3 REST API而不是尝试将存储桶安装为卷,但现在看来这RioFS可能是一个可行的选择。

采纳答案by BraveNewCurrency

Have you ever mounted an S3 bucket on a local linux filesystem?

您是否曾经在本地 linux 文件系统上挂载过 S3 存储桶?

No. It's fun for testing, but I wouldn't let it near a production system. It's much better to use a library to communicate with S3. Here's why:

不。测试很有趣,但我不会让它靠近生产系统。使用库与 S3 通信要好得多。原因如下:

  1. It won't hide errors. A filesystem only has a few errors codes it can send you to indicate a problem. An S3 library will give you the exact error message from Amazon so you understand what's going on, log it, handle corner cases, etc.
  2. A library will use less memory. Filesystems layers will cache lots of random stuff that you many never use again. A library puts you in control to decide what to cache and not to cache.
  3. Expansion. If you ever need to do anything fancy (set an ACL on a file, generate a signed link, versioning, lifecycle, change durability, etc), then you'll have to dump your filesystem abstraction and use a library anyway.
  4. Timing and retries. Some fraction of requests randomly error out and can be retried. Sometimes you may want to retry a lot, sometimes you would rather error out quickly. A filesystem doesn't give you granular control, but a library will.
  1. 它不会隐藏错误。文件系统只有一些错误代码,它可以发送给您以指示问题。S3 库将为您提供来自 Amazon 的确切错误消息,以便您了解正在发生的事情、记录它、处理极端情况等。
  2. 库将使用更少的内存。文件系统层将缓存许多您永远不会再使用的随机内容。库让您可以控制决定缓存和不缓存的内容。
  3. 扩张。如果您需要做任何花哨的事情(在文件上设置 ACL、生成签名链接、版本控制、生命周期、更改持久性等),那么您将不得不转储文件系统抽象并使用库。
  4. 计时和重试。一部分请求随机出错,可以重试。有时您可能想多次重试,有时您宁愿快速出错。文件系统不会给你细粒度的控制,但库会。

The bottom line is that S3 under FUSE is a leaky abstraction. S3 doesn't have (or need) directories. Filesystems weren't built for billions of files. Their permissions models are incompatible. You are wasting a lot of the power of S3 by trying to shoehorn it into a filesystem.

底线是 FUSE 下的 S3 是一个有漏洞的抽象。S3 没有(或不需要)目录。文件系统不是为数十亿个文件构建的。他们的权限模型不兼容。您试图将 S3 硬塞到文件系统中,浪费了 S3 的大量功能。

Two random PHP libraries for talking to S3:

用于与 S3 对话的两个随机 PHP 库:

https://github.com/KnpLabs/Gaufrette

https://github.com/KnpLabs/Gaufrette

https://aws.amazon.com/sdkforphp/- this one is useful if you expand beyond just using S3, or if you need to do any of the fancy requests mentioned above.

https://aws.amazon.com/sdkforphp/- 如果您扩展到不只是使用 S3,或者您需要执行上述任何奇特的请求,则此方法很有用。

回答by PlayGod

Quite often, it is advantageous to write files to the EBS volume, then force subsequent public requests for the file(s) to route through CloudFront CDN.

通常,将文件写入 EBS 卷是有利的,然后强制对文件的后续公共请求通过 CloudFront CDN 进行路由。

In that way, if the app must do any transformations to the file, it's much easier to do on the local drive & system, then force requests for the transformed files to pull from the origin via CloudFront.

这样,如果应用程序必须对文件进行任何转换,那么在本地驱动器和系统上执行要容易得多,然后强制请求转换后的文件通过 CloudFront 从源中提取。

e.g. if your user is uploading an image for an avatar, and the avatar image needs several iterations for size & crop, your app can create these on the local volume, but all public requests for the file will take place through a cloudfront origin-pull request. In that way, you have maximum flexibility to keep the original file (or an optimized version of the file), and any subsequent user requests can either pull an existing version from cloud front edge, or cloud front will route the request back to the app and create any necessary iterations.

例如,如果您的用户正在上传头像的图像,并且头像图像需要多次迭代来调整大小和裁剪,您的应用程序可以在本地卷上创建这些,但对文件的所有公开请求都将通过云前端原点拉取进行要求。这样,您可以最大程度地灵活地保留原始文件(或文件的优化版本),并且任何后续用户请求都可以从云前端拉取现有版本,或者云前端会将请求路由回应用程序并创建任何必要的迭代。

An elementary example of the above would be WordPress, which creates multiple sized/cropped versions of any graphic image uploaded, in addition to keeping the original (subject to file size restrictions, and/or plugin transformations). CDN-capable WordPress plugins such as W3 Total Cache rewrite requests to pull through CDN, so the app only needs to create unique first-request iterations. Adding browser caching URL versioning (http://domain.tld/file.php?x123) further refines and leverages CDN functionality.

上面的一个基本示例是 WordPress,除了保留原始图像(受文件大小限制和/或插件转换的影响)之外,它还会为上传的任何图形图像创建多个大小/裁剪版本。支持 CDN 的 WordPress 插件(例如 W3 Total Cache)重写请求以拉取 CDN,因此应用程序只需要创建唯一的第一次请求迭代。添加浏览器缓存 URL 版本控制 ( http://domain.tld/file.php?x123) 进一步优化和利用 CDN 功能。

If you are concerned about rapid expansion of EBS volume file size or inodes, you can automate a pruning process for seldom-requested files, or aged files.

如果您担心 EBS 卷文件大小或 inode 的快速扩展,您可以为很少请求的文件或陈旧的文件自动执行修剪过程。