C# WCF 在自托管服务上流式传输大数据 (500MB / 1GB)
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/14479885/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
WCF Streaming large data (500MB / 1GB) on a self hosted service
提问by user2003667
I'm currently experiencing an issue trying to send large data using WCF self hosted service (no IIS). Transferring 500MB using streaming results with my service crashing with System.OutOfMemoryException. Is it possible at all to transfer such amount of data?
我目前在尝试使用 WCF 自托管服务（无 IIS）发送大数据时遇到问题。使用流结果传输 500MB，我的服务因 System.OutOfMemoryException 崩溃。是否有可能传输如此大量的数据？
Here is my WCF configuration:
这是我的 WCF 配置：
<system.serviceModel>
<services>
<service name="CIService" behaviorConfiguration="CIBehavior">
<host>
<baseAddresses>
<add baseAddress="net.tcp://localhost:6547/CIService/CIService.svc" />
</baseAddresses>
</host>
<endpoint binding="netTcpBinding"
bindingConfiguration="netTcpBindingConfig"
behaviorConfiguration="CIBehavior.EndpointBehavior"
contract="CIService.ICreatable" />
<endpoint address="mex"
binding="mexHttpBinding"
name="mexTcpBinding"
contract="IMetadataExchange" />
</service>
</services>
<serviceHostingEnvironment multippleSiteBindingEnabled="True" />
<bindings>
<netTcpBinding>
<binding name="netTcpBindingConfig" closeTimeout="00:01:00" openTimeout="00:01:00"
receiveTimeout="01:00:00" sendTimeout="00:10:00"
hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxConnections="10"
maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"
transferMode="Streamed">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647"
maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
</binding>
</netTcpBinding>
</bindings>
<behaviors>
<serviceBehaviors>
<behavior name="CIBehavior">
<serviceMetadata httpGetEnabled="False" />
<serviceDebug includeExceptionDetailInFaults="true" />
<serviceThrottling maxConcurrentCalls="200" maxConcurrentInstances="2147483647" maxConcurrentSessions="100" />
<dataContractSerializer maxItemsInObjectGraph="2147483647"/>
</behavior>
</serviceBehaviors>
<endpointBehavior>
<behavior name="CIBehavior.EndpointBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</endpointBehavior>
</behaviors>
</system.serviceModel>
My client configuration:
我的客户端配置：
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="NetTcpBinding_ICreatable"
closeTimeout="00:01:00" openTimeout="00:01:00"
receiveTimeout="01:00:00" sendTimeout="00:10:00"
transactionFlow="false"
transferMode="Streamed"
transactionProtocol="OleTransactions"
hostNameComparisonMode="StrongWildcard"
listenBacklog="10"
maxBufferPoolSize="2147483647"
maxBufferSize="2147483647"
maxConnections="10"
maxReceivedMessageSize ="2147483647">
<readerQuotas
maxDepth="2147483647"
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647" />
<reliableSession ordered="true" inactivityTimeout="00:10:00" anabled="false" />
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint name="NetTcpBinding_ICreatable"
address="net.tcp://localhost:6547/CIService/CIService.svc"
binding="netTcpBinding"
bindingConfiguration="NetTcpBinding_ICreatable"
behaviorConfiguration="CIBehavior.EndpointBehavior"
contract="ICreatable" />
</client>
<behaviors>
<endpointBehavior>
<behavior name="CIBehavior.EndpointBehavior">
<dataContractSerializer maxItemsInObjectGraph="2147483647" />
</behavior>
</endpointBehavior>
</behaviors>
</system.serviceModel>
回答by DerekGn
You dont need maxBufferSizeor maxBufferPoolSizeset so high these are possibly causing your out of memory exception. The defaults should be fine.
您不需要将maxBufferSize或maxBufferPoolSize设置得太高，这些可能会导致您的内存不足异常。默认值应该没问题。
Check out Large Data and Streamingon MSDN, specifically the section Special Security Considerations for Large Datathis piece of the text is important
查看MSDN 上的大数据和流，特别是大数据的特殊安全注意事项部分，这部分文本很重要
The MaxBufferSize property is required to constrain the memory that WCF buffers. It is important to set this to a safe value (or keep it at the default value) when streaming. For example, suppose your service must receive files up to 4 GB in size and store them on the local disk. Suppose also that your memory is constrained in such a way that you can only buffer 64 KB of data at a time. Then you would set the MaxReceivedMessageSize to 4 GB and MaxBufferSize to 64 KB. Also, in your service implementation, you must ensure that you read only from the incoming stream in 64-KB chunks and do not read the next chunk before the previous one has been written to disk and discarded from memory.
需要 MaxBufferSize 属性来限制 WCF 缓冲的内存。流式传输时将其设置为安全值（或保持默认值）很重要。例如，假设您的服务必须接收最大 4 GB 的文件并将它们存储在本地磁盘上。还假设您的内存受到限制，以至于您一次只能缓冲 64 KB 的数据。然后将 MaxReceivedMessageSize 设置为 4 GB，将 MaxBufferSize 设置为 64 KB。此外，在您的服务实现中，您必须确保仅从 64 KB 块中的传入流中读取，并且在前一个块已写入磁盘并从内存中丢弃之前不要读取下一个块。
I put together a very simple example of streaming data from a self hosted service to a console client. To keep the post short I only added the server code and part of the client.
我整理了一个非常简单的示例，将数据从自托管服务流式传输到控制台客户端。为了保持帖子简短，我只添加了服务器代码和客户端的一部分。
The service contract
服务合同
using System.IO;
using System.ServiceModel;
namespace Service
{
[ServiceContract]
public interface IStream
{
[OperationContract]
Stream GetLargeObject();
}
}
The service implementation
服务实施
using System;
using System.IO;
using System.ServiceModel;
namespace Service
{
[ServiceBehavior]
public class StreamService : IStream
{
public Stream GetLargeObject()
{
// Add path to a big file, this one is 2.5 gb
string filePath = Path.Combine(Environment.CurrentDirectory, "C:\Temp\BFBC2_PC_Client_R11_795745_Patch.exe");
try
{
FileStream imageFile = File.OpenRead(filePath);
return imageFile;
}
catch (IOException ex)
{
Console.WriteLine(String.Format("An exception was thrown while trying to open file {0}", filePath));
Console.WriteLine("Exception is: ");
Console.WriteLine(ex.ToString());
throw;
}
}
}
}
The service main
服务主
using System;
using System.ServiceModel;
namespace Service
{
class Program
{
static void Main(string[] args)
{
try
{
using (var serviceHost = new ServiceHost(typeof(StreamService)))
{
serviceHost.Open();
Console.WriteLine("Press Any Key to end");
Console.ReadKey();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
}
}
The service app.config
服务 app.config
<?xml version="1.0"?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
<system.serviceModel>
<behaviors>
<serviceBehaviors>
<behavior name="StreamServiceBehavior">
<serviceMetadata httpGetEnabled="True" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netTcpBinding>
<binding name="NewBinding0" transferMode="Streamed"/>
</netTcpBinding>
</bindings>
<services>
<service behaviorConfiguration="StreamServiceBehavior" name="Service.StreamService">
<endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
bindingConfiguration="NewBinding0" bindingName="" contract="Service.IStream" />
<endpoint address="mex" binding="mexHttpBinding"
contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="http://localhost:8080/StreamService" />
</baseAddresses>
</host>
</service>
</services>
</system.serviceModel>
</configuration>
Launch the service, may need to run under admin account to open the socket. Create a client console application and add a service reference using the url http:// localhost:8080 / StreamService, using Service as the namespace for the generated client.
启动服务，可能需要在admin账户下运行才能打开socket。创建客户端控制台应用程序并使用 url http://localhost:8080/StreamService 添加服务引用，使用 Service 作为生成的客户端的命名空间。
The client main
客户主要
using System;
using System.IO;
using Client.Service;
namespace Client
{
class Program
{
static void Main(string[] args)
{
try
{
using (StreamClient streamClient = new StreamClient())
{
streamClient.Open();
using (FileStream fileStream = new FileStream("c:\temp\bigfile.exe",FileMode.Create))
{
streamClient.GetLargeObject().CopyTo(fileStream);
}
}
Console.WriteLine("Press any key to end");
Console.ReadKey();
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
}
}
The generated client config file will need to be modified slightly, increase receiveTimeoutand set maxReceivedMessageSize="4294967295"
生成的客户端配置文件需要稍作修改，增加receiveTimeout并设置maxReceivedMessageSize="4294967295"
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding name="NetTcpBinding_IStream" closeTimeout="00:01:00"
openTimeout="00:01:00" receiveTimeout="00:30:00" sendTimeout="00:01:00"
transactionFlow="false" transferMode="Streamed" transactionProtocol="OleTransactions"
hostNameComparisonMode="StrongWildcard" listenBacklog="10"
maxBufferPoolSize="524288" maxBufferSize="65536" maxConnections="10"
maxReceivedMessageSize="4294967295">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<reliableSession ordered="true" inactivityTimeout="00:10:00"
enabled="false" />
<security mode="Transport">
<transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
<message clientCredentialType="Windows" />
</security>
</binding>
</netTcpBinding>
</bindings>
<client>
<endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
bindingConfiguration="NetTcpBinding_IStream" contract="Service.IStream"
name="NetTcpBinding_IStream">
</endpoint>
</client>
</system.serviceModel>
Launch the service then the client. It will stream a large file without issue.
启动服务，然后启动客户端。它将毫无问题地流式传输大文件。
