如何使用 Java REST 服务和数据流下载文件

How to download a file using a Java REST service and a data stream

提问by Alex

I have 3 machines:

1. server where the file is located
   
2. server where REST service is running ( Jersey)
   
3. client(browser) with access to 2nd server but no access to 1st server

我有3台机器：

1. 文件所在的服务器
   
2. 运行 REST 服务的服务器（Jersey）
   
3. 客户端（浏览器）可以访问第二台服务器但无法访问第一台服务器

How can I directly (without saving the file on 2nd server) download the file from 1st server to client's machine?
From 2nd server I can get a ByteArrayOutputStreamto get the file from 1st server, can I pass this stream further to the client using the REST service?

如何直接（不将文件保存在第二台服务器上）将文件从第一台服务器下载到客户端机器？
我可以从第二台服务器获取ByteArrayOutputStream以从第一台服务器获取文件，我可以使用 REST 服务将此流进一步传递给客户端吗？

It will work this way?

它会这样工作吗？

So basically what I want to achieve is to allow the client to download a file from 1st server using the REST service on 2nd server (since there is no direct access from client to 1st server) using only data streams (so no data touching the file system of 2nd server).

所以基本上我想要实现的是允许客户端使用第二台服务器上的 REST 服务从第一台服务器下载文件（因为没有从客户端到第一台服务器的直接访问）仅使用数据流（因此没有数据接触文件第二个服务器的系统）。

What I try now with EasyStream library:

我现在尝试使用 EasyStream 库：

       final FTDClient client = FTDClient.getInstance();

        try {
            final InputStreamFromOutputStream<String> isOs = new InputStreamFromOutputStream<String>() {
                @Override
                public String produce(final OutputStream dataSink) throws Exception {
                    return client.downloadFile2(location, Integer.valueOf(spaceId), URLDecoder.decode(filePath, "UTF-8"), dataSink);
                }
            };
            try {
                String fileName = filePath.substring(filePath.lastIndexOf("/") + 1);

                StreamingOutput output = new StreamingOutput() {
                    @Override
                    public void write(OutputStream outputStream) throws IOException, WebApplicationException {
                        int length;
                        byte[] buffer = new byte[1024];
                        while ((length = isOs.read(buffer)) != -1){
                            outputStream.write(buffer, 0, length);
                        }
                        outputStream.flush();

                    }
                };
                return Response.ok(output, MediaType.APPLICATION_OCTET_STREAM)
                        .header("Content-Disposition", "attachment; filename=\"" + fileName + "\"" )
                        .build();

UPDATE2

更新2

So my code now with the custom MessageBodyWriter looks simple:

所以我现在使用自定义 MessageBodyWriter 的代码看起来很简单：

ByteArrayOutputStream baos = new ByteArrayOutputStream(2048) ;
client.downloadFile(location, spaceId, filePath, baos);
return Response.ok(baos).build();

But I get the same heap error when trying with large files.

但是在尝试处理大文件时，我遇到了相同的堆错误。

UPDATE3Finally managed to get it working ! StreamingOutput did the trick.

UPDATE3终于设法让它工作了！StreamingOutput 做到了。

Thank you @peeskillet ! Many thanks !

谢谢@peeskillet！非常感谢 ！

采纳答案by Paul Samsotha

"How can I directly (without saving the file on 2nd server) download the file from 1st server to client's machine?"

“我如何直接（不将文件保存在第二台服务器上）将文件从第一台服务器下载到客户端的机器上？”

Just use the ClientAPI and get the InputStreamfrom the response

只需使用ClientAPI 并InputStream从响应中获取

Client client = ClientBuilder.newClient();
String url = "...";
final InputStream responseStream = client.target(url).request().get(InputStream.class);

There are two flavors to get the InputStream. You can also use

有两种口味可以得到InputStream。你也可以使用

Response response = client.target(url).request().get();
InputStream is = (InputStream)response.getEntity();

Which one is the more efficient? I'm not sure, but the returned InputStreams are different classes, so you may want to look into that if you care to.

哪个效率更高？我不确定，但返回的InputStreams 是不同的类，所以如果你愿意，你可能想研究一下。

From 2nd server I can get a ByteArrayOutputStream to get the file from 1st server, can I pass this stream further to the client using the REST service?

我可以从第二台服务器获取 ByteArrayOutputStream 以从第一台服务器获取文件，我可以使用 REST 服务将此流进一步传递给客户端吗？

So most of the answers you'll see in the link provided by @GradyGCooperseem to favor the use of StreamingOutput. An example implementation might be something like

因此，您将在@GradyGCooper 提供的链接中看到的大多数答案似乎都倾向于使用StreamingOutput. 示例实现可能类似于

final InputStream responseStream = client.target(url).request().get(InputStream.class);
System.out.println(responseStream.getClass());
StreamingOutput output = new StreamingOutput() {
    @Override
    public void write(OutputStream out) throws IOException, WebApplicationException {  
        int length;
        byte[] buffer = new byte[1024];
        while((length = responseStream.read(buffer)) != -1) {
            out.write(buffer, 0, length);
        }
        out.flush();
        responseStream.close();
    }   
};
return Response.ok(output).header(
        "Content-Disposition", "attachment, filename=\"...\"").build();

But if we look at the source code for StreamingOutputProvider, you'll see in the writeTo, that it simply writes the data from one stream to another. So with our implementation above, we have to write twice.

但是，如果我们查看StreamingOutputProvider的源代码，您会在 中看到writeTo，它只是将数据从一个流写入另一个流。所以对于我们上面的实现，我们必须写两次。

How can we get only one write? Simple return the InputStreamas the Response

我们怎么能只写一个？简单返回InputStream作为Response

final InputStream responseStream = client.target(url).request().get(InputStream.class);
return Response.ok(responseStream).header(
        "Content-Disposition", "attachment, filename=\"...\"").build();

If we look at the source code for InputStreamProvider, it simply delegates to ReadWriter.writeTo(in, out), which simply does what we did above in the StreamingOutputimplementation

如果我们查看InputStreamProvider的源代码，它只是委托给ReadWriter.writeTo(in, out)，它只是执行我们在上面的StreamingOutput实现中所做的

 public static void writeTo(InputStream in, OutputStream out) throws IOException {
    int read;
    final byte[] data = new byte[BUFFER_SIZE];
    while ((read = in.read(data)) != -1) {
        out.write(data, 0, read);
    }
}

Asides:

旁白：

• Clientobjects are expensive resources. You may want to reuse the same Clientfor request. You can extract a WebTargetfrom the client for each request.

  WebTarget target = client.target(url);
  InputStream is = target.request().get(InputStream.class);
  

  I think the WebTargetcan even be shared. I can't find anything in the Jersey 2.x documentation(only because it is a larger document, and I'm too lazy to scan through it right now :-), but in the Jersey 1.x documentation, it says the Clientand WebResource(which is equivalent to WebTargetin 2.x) can be shared between threads. So I'm guessing Jersey 2.x would be the same. but you may want to confirm for yourself.

• You don't have to make use of the ClientAPI. A download can be easily achieved with the java.netpackage APIs. But since you're already using Jersey, it doesn't hurt to use its APIs

• The above is assuming Jersey 2.x. For Jersey 1.x, a simple Google search should get you a bunch of hits for working with the API (or the documentation I linked to above)

• Client对象是昂贵的资源。您可能希望Client对请求重复使用相同的内容。您可以WebTarget为每个请求从客户端提取一个。

  WebTarget target = client.target(url);
  InputStream is = target.request().get(InputStream.class);
  

  我认为WebTarget甚至可以共享。我在Jersey 2.x 文档中找不到任何内容（只是因为它是一个较大的文档，我现在懒得浏览它:-)，但是在Jersey 1.x 文档中，它说Client和WebResource（相当于WebTarget在 2.x 中）可以在线程之间共享。所以我猜 Jersey 2.x 会是一样的。但你可能想自己确认一下。

• 您不必使用ClientAPI。使用java.net包 API可以轻松实现下载。但既然你已经在使用 Jersey，那么使用它的 API 并没有什么坏处

• 以上是假设 Jersey 2.x。对于 Jersey 1.x，一个简单的 Google 搜索应该会为您提供大量使用 API（或我在上面链接到的文档）的点击量

---

---

UPDATE

更新

I'm such a dufus. While the OP and I are contemplating ways to turn a ByteArrayOutputStreamto an InputStream, I missed the simplest solution, which is simply to write a MessageBodyWriterfor the ByteArrayOutputStream

我就是个笨蛋。虽然 OP 和我正在考虑将 a 转换ByteArrayOutputStream为 an 的方法InputStream，但我错过了最简单的解决方案，即简单地MessageBodyWriter为ByteArrayOutputStream

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.lang.annotation.Annotation;
import java.lang.reflect.Type;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.ext.MessageBodyWriter;
import javax.ws.rs.ext.Provider;

@Provider
public class OutputStreamWriter implements MessageBodyWriter<ByteArrayOutputStream> {

    @Override
    public boolean isWriteable(Class<?> type, Type genericType,
            Annotation[] annotations, MediaType mediaType) {
        return ByteArrayOutputStream.class == type;
    }

    @Override
    public long getSize(ByteArrayOutputStream t, Class<?> type, Type genericType,
            Annotation[] annotations, MediaType mediaType) {
        return -1;
    }

    @Override
    public void writeTo(ByteArrayOutputStream t, Class<?> type, Type genericType,
            Annotation[] annotations, MediaType mediaType,
            MultivaluedMap<String, Object> httpHeaders, OutputStream entityStream)
            throws IOException, WebApplicationException {
        t.writeTo(entityStream);
    }
}

Then we can simply return the ByteArrayOutputStreamin the response

然后我们可以简单地ByteArrayOutputStream在响应中返回

return Response.ok(baos).build();

D'OH!

哦！

UPDATE 2

更新 2

Here are the tests I used (

这是我使用的测试（

Resource class

资源类

@Path("test")
public class TestResource {

    final String path = "some_150_mb_file";

    @GET
    @Produces(MediaType.APPLICATION_OCTET_STREAM)
    public Response doTest() throws Exception {
        InputStream is = new FileInputStream(path);
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        int len;
        byte[] buffer = new byte[4096];
        while ((len = is.read(buffer, 0, buffer.length)) != -1) {
            baos.write(buffer, 0, len);
        }
        System.out.println("Server size: " + baos.size());
        return Response.ok(baos).build();
    }
}

Client test

客户端测试

public class Main {
    public static void main(String[] args) throws Exception {
        Client client = ClientBuilder.newClient();
        String url = "http://localhost:8080/api/test";
        Response response = client.target(url).request().get();
        String location = "some_location";
        FileOutputStream out = new FileOutputStream(location);
        InputStream is = (InputStream)response.getEntity();
        int len = 0;
        byte[] buffer = new byte[4096];
        while((len = is.read(buffer)) != -1) {
            out.write(buffer, 0, len);
        }
        out.flush();
        out.close();
        is.close();
    }
}

UPDATE 3

更新 3

So the final solution for this particular use case was for the OP to simply pass the OutputStreamfrom the StreamingOutput's writemethod. Seems the third-party API, required a OutputStreamas an argument.

所以这个特定用例的最终解决方案是让 OP 简单地传递OutputStreamfromStreamingOutput的write方法。似乎是第三方 API，需要 aOutputStream作为参数。

StreamingOutput output = new StreamingOutput() {
    @Override
    public void write(OutputStream out) {
        thirdPartyApi.downloadFile(.., .., .., out);
    }
}
return Response.ok(output).build();

Not quite sure, but seems the reading/writing within the resource method, using ByteArrayOutputStream`, realized something into memory.

不太确定，但似乎资源方法中的读/写，使用 ByteArrayOutputStream`，在内存中实现了一些东西。

The point of the downloadFilemethod accepting an OutputStreamis so that it can write the result directly to the OutputStreamprovided. For instance a FileOutputStream, if you wrote it to file, while the download is coming in, it would get directly streamed to the file.

downloadFile接受 an的方法的重点OutputStream是它可以将结果直接写入所OutputStream提供的。例如 a FileOutputStream，如果您将其写入文件，则在下载进入时，它将直接流式传输到文件中。

It's not meant for us to keep a reference to the OutputStream, as you were trying to do with the baos, which is where the memory realization comes in.

这并不意味着我们要保留对 的引用OutputStream，就像您试图对 做的那样baos，这是内存实现的来源。

So with the way that works, we are writing directly to the response stream provided for us. The method writedoesn't actually get called until the writeTomethod (in the MessageBodyWriter), where the OutputStreamis passed to it.

因此，通过这种方式，我们直接写入为我们提供的响应流。该方法write实际上不会被调用，直到writeTo方法（在 中MessageBodyWriter）OutputStream传递给它。

You can get a better picture looking at the MessageBodyWriterI wrote. Basically in the writeTomethod, replace the ByteArrayOutputStreamwith StreamingOutput, then inside the method, call streamingOutput.write(entityStream). You can see the link I provided in the earlier part of the answer, where I link to the StreamingOutputProvider. This is exactly what happens

您可以通过查看MessageBodyWriter我写的内容获得更好的图片。基本上是在writeTo方法中，替换ByteArrayOutputStream使用StreamingOutput，则内部的方法，呼叫streamingOutput.write(entityStream)。您可以看到我在答案前面部分提供的链接，我在其中链接到StreamingOutputProvider. 这正是发生的事情

回答by Grady G Cooper

See example here: Input and Output binary streams using JERSEY?

请参见此处的示例：使用 JERSEY 的输入和输出二进制流？

Pseudo code would be something like this (there are a few other similar options in above mentioned post):

伪代码将是这样的（上面提到的帖子中还有一些其他类似的选项）：

@Path("file/")
@GET
@Produces({"application/pdf"})
public StreamingOutput getFileContent() throws Exception {
     public void write(OutputStream output) throws IOException, WebApplicationException {
        try {
          //
          // 1. Get Stream to file from first server
          //
          while(<read stream from first server>) {
              output.write(<bytes read from first server>)
          }
        } catch (Exception e) {
            throw new WebApplicationException(e);
        } finally {
              // close input stream
        }
    }
}

回答by AVINASH SHRIMALI

Refer this:

参考这个：

@RequestMapping(value="download", method=RequestMethod.GET)
public void getDownload(HttpServletResponse response) {

// Get your file stream from wherever.
InputStream myStream = someClass.returnFile();

// Set the content type and attachment header.
response.addHeader("Content-disposition", "attachment;filename=myfilename.txt");
response.setContentType("txt/plain");

// Copy the stream to the response's output stream.
IOUtils.copy(myStream, response.getOutputStream());
response.flushBuffer();
}

Details at: https://twilblog.github.io/java/spring/rest/file/stream/2015/08/14/return-a-file-stream-from-spring-rest.html

详情见：https: //twilblog.github.io/java/spring/rest/file/stream/2015/08/14/return-a-file-stream-from-spring-rest.html