Friday, March 1, 2013

Download large file using WCF


Recently, we were facing issues when we need to download more than 2GB file like 4GB, 5GB etc from WCF service.

It worked till 2GB by default because max value permitted in maxReceviedBufferSize=int32.MaxValue which is 2147483647 (1.99 GB). What if we need to download file more than 2GB??  If you try, you will get following error:

"The maximum message size quota for incoming messages (2147483647 ) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element"

After searching on internet, following options suggested but none worked:

1. Only change transfermode to Stream.
2. Set Max value of all elements like

<binding name="BasicHttpBinding_Service" messageEncoding="Mtom" maxReceivedMessageSize="2147483647" maxBufferPoolSize="2147483647" maxBufferSize="2147483647" transferMode="Streamed"  useDefaultWebProxy="true" sendTimeout="05:00:00" closeTimeout="05:00:00"/>

Finally, I got a way to work after tweaking the client config as follows:

<binding name="BasicHttpBinding_Service" messageEncoding="Mtom" maxReceivedMessageSize="4294967294" maxBufferPoolSize="65536" maxBufferSize="65536" transferMode="Streamed"  useDefaultWebProxy="true" sendTimeout="05:00:00" closeTimeout="05:00:00"/>

If you note here, we have change following property as follows:
1. maxReceivedMessageSize="4294967294"  which is 4 GB
2.  maxBufferPoolSize = "65536" which is 64 KB
3. maxBufferSize="65536" which is 64 KB
4. transferMode = "Streamed"

Restricting the maximum incoming message size is not enough in this case. The MaxBufferSize property is required to constrain the memory that WCF buffers. It is important to set this to a safe value (or keep it at the default value) when streaming. For example, suppose your service must receive files up to 4 GB in size and store them on the local disk. Suppose also that your memory is constrained in such a way that you can only buffer 64 KB of data at a time. Then you would set the MaxReceivedMessageSize to 4 GB and MaxBufferSize to 64 KB. Also, in your service implementation, you must ensure that you read only from the incoming stream in 64-KB chunks and do not read the next chunk before the previous one has been written to disk and discarded from memory

Note
1.As it opens Stream to your client, pay a special attention about writing this stream. I would recommend not to  use MemoryStream to consume same because it is a large file and use of MemoryStream will kill you box and effect system performance. If you are using a web application, multiple users can try to download same file which can exhaust your box.

2. As a solution, download file to a temporary location and play as per your need for better for performance.

3. If you have any requirement to download more than 4 GB, you can change to maxReceivedMessageSize value as per your need and let other values unchanged.

Hope this helps who have similar type of problem. I'll be happy to listen your feedback/comment that can help me to improve.

Thanks!
Tarun