Wcf System.InsufficientMemoryException. Nie udało się przydzielić zarządzanego bufora pamięci o wielkości 536870912 bajtów

Question

Mam metodę usługi WCF, która zwraca dużą tablicę bajtów [] o rozmiarze około 2 mb. Jednak po uruchomieniu tej metody z WcfTestClient, otrzymuję następujący wyjątek.

System.InsufficientMemoryException 
Failed to allocate a managed memory buffer of 536870912 bytes. The amount of available memory may be low. 

Szukałem na google, a ludzie wydają się mieć ten problem, ale mam trudności ze znalezieniem jakiegokolwiek rozwiązania i rzeczywistą przyczynę, dlaczego jest ona stara się stworzyć taki ogromny bufor w pierwszej kolejności?

Poniżej znajdują się moje ustawienia konfiguracji. Ustawiłem maxItemsInObjectGraph = "2147483647", w przeciwnym razie był to wyjątek. Reszta jest prosta.

Server

<system.serviceModel> 
    <services> 
     <service behaviorConfiguration="Wcf.ServiceBehavior" 
     name="Wcf.WcfService"> 
     <endpoint address="" binding="wsHttpBinding" contract="Wcf.IWcfService"> 
      <identity> 
      <dns value="localhost" /> 
      </identity> 
     </endpoint> 
     <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" /> 
     </service> 
    </services> 
    <behaviors> 
     <serviceBehaviors> 
     <behavior name="Wcf.ServiceBehavior"> 
      <serviceMetadata httpGetEnabled="true" /> 
      <serviceDebug includeExceptionDetailInFaults="true" /> 
      <dataContractSerializer maxItemsInObjectGraph="2147483647" /> 
     </behavior> 
     </serviceBehaviors> 
    </behaviors> 

    <diagnostics>  
     <messageLogging maxMessagesToLog="30000"  
       logEntireMessage="true"  
       logMessagesAtServiceLevel="true"  
       logMalformedMessages="true"  
       logMessagesAtTransportLevel="true">  
     </messageLogging>  
    </diagnostics> 
    </system.serviceModel> 

WcfTestClient Config To jest domyślny plik konfiguracyjny, który jest ładowany z WcfTestClient

<?xml version="1.0" encoding="utf-8"?> 
<configuration> 
    <system.serviceModel> 
     <bindings> 
      <wsHttpBinding> 
       <binding name="WSHttpBinding_IWcfService" closeTimeout="00:01:00" 
        openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00" 
        bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard" 
        maxBufferPoolSize="524288" maxReceivedMessageSize="65536" 
        messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true" 
        allowCookies="false"> 
        <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384" 
         maxBytesPerRead="4096" maxNameTableCharCount="16384" /> 
        <reliableSession ordered="true" inactivityTimeout="00:10:00" 
         enabled="false" /> 
        <security mode="Message"> 
         <transport clientCredentialType="Windows" proxyCredentialType="None" 
          realm="" /> 
         <message clientCredentialType="Windows" negotiateServiceCredential="true" 
          algorithmSuite="Default" /> 
        </security> 
       </binding> 
      </wsHttpBinding> 
     </bindings> 
     <client> 
      <endpoint address="http://localhost:4151/Service1.svc" binding="wsHttpBinding" 
       bindingConfiguration="WSHttpBinding_IWcfService" contract="IWcfService" 
       name="WSHttpBinding_IWcfService"> 
       <identity> 
        <dns value="localhost" /> 
       </identity> 
      </endpoint> 
     </client> 
    </system.serviceModel> 
</configuration> 

Oto wyjątek Śledzenie stosu

<E2ETraceEvent xmlns="http://schemas.microsoft.com/2004/06/E2ETraceEvent"> 

    <System xmlns="http://schemas.microsoft.com/2004/06/windows/eventlog/system"> 

    <EventID>131075</EventID> 
    <Type>3</Type> 
    <SubType Name="Error">0</SubType> 
    <Level>2</Level> 
    <TimeCreated SystemTime="2010-06-29T09:18:23.9616589Z" /> 
    <Source Name="System.ServiceModel" /> 
    <Correlation ActivityID="{7870ff09-e268-4e9d-a692-389fd03db1aa}" /> 
    <Execution ProcessName="WebDev.WebServer20" ProcessID="4812" 
    ThreadID="17" /> 
    <Channel /> 
    <Computer>PC-008915</Computer> 
    </System> 
    <ApplicationData> 
    <TraceData> 
     <DataItem> 
     <TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" 
     Severity="Error"> 
      <TraceIdentifier> 
      http://msdn.microsoft.com/en-GB/library/System.ServiceModel.Diagnostics.ThrowingException.aspx</TraceIdentifier> 
      <Description>Throwing an exception.</Description> 
      <AppDomain>5ab4f443-1-129222741421051645</AppDomain> 
      <Exception> 
      <ExceptionType>System.InsufficientMemoryException, 
      mscorlib, Version=2.0.0.0, Culture=neutral, 
      PublicKeyToken=b77a5c561934e089</ExceptionType> 
      <Message>Failed to allocate a managed memory buffer of 
      536870912 bytes. The amount of available memory may be 
      low.</Message> 
      <StackTrace>at 
      System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32 
      size) at 
      System.ServiceModel.Channels.BufferManager.GCBufferManager.TakeBuffer(Int32 
      bufferSize) at 
      System.ServiceModel.Channels.BufferedOutputStream.AllocNextChunk(Int32 
      minimumChunkSize) at 
      System.ServiceModel.Channels.BufferedOutputStream.WriteCore(Byte[] 
      buffer, Int32 offset, Int32 size) at 
      System.ServiceModel.Channels.BufferedOutputStream.Write(Byte[] 
      buffer, Int32 offset, Int32 size) at 
      System.Xml.XmlStreamNodeWriter.FlushBuffer() at 
      System.Xml.XmlStreamNodeWriter.GetBuffer(Int32 count, 
      Int32&amp; offset) at 
      System.Xml.XmlStreamNodeWriter.UnsafeWriteUTF8Chars(Char* 
      chars, Int32 charCount) at 
      System.Xml.XmlStreamNodeWriter.WriteUTF8Chars(String 
      value) at 
      System.Xml.XmlUTF8NodeWriter.WriteEndElement(String 
      prefix, String localName) at 
      System.Xml.XmlSigningNodeWriter.WriteEndElement(String 
      prefix, String localName) at 
      System.Xml.XmlBaseWriter.WriteEndElement() at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteString(XmlWriterDelegator 
      xmlWriter, String value, XmlDictionaryString name, 
      XmlDictionaryString ns) at 
      WritePropertyDtoToXml(XmlWriterDelegator , Object , 
      XmlObjectSerializerWriteContext , ClassDataContract) 
      at 
      System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator 
      xmlWriter, Object obj, XmlObjectSerializerWriteContext 
      context) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator 
      xmlWriter, Object obj, Boolean isDeclaredType, Boolean 
      writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle 
      declaredTypeHandle) at 
      WriteArrayOfPropertyDtoToXml(XmlWriterDelegator , 
      Object , XmlObjectSerializerWriteContext , 
      CollectionDataContract) at 
      System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(XmlWriterDelegator 
      xmlWriter, Object obj, XmlObjectSerializerWriteContext 
      context) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator 
      xmlWriter, Object obj, Boolean isDeclaredType, Boolean 
      writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle 
      declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator 
      xmlWriter, Object obj, Boolean isDeclaredType, Boolean 
      writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle 
      declaredTypeHandle) at 
      WriteEntityTypeDtoToXml(XmlWriterDelegator , Object , 
      XmlObjectSerializerWriteContext , ClassDataContract) 
      at 
      System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator 
      xmlWriter, Object obj, XmlObjectSerializerWriteContext 
      context) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator 
      xmlWriter, Object obj, Boolean isDeclaredType, Boolean 
      writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle 
      declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(XmlWriterDelegator 
      xmlWriter, Object obj, Boolean isDeclaredType, Boolean 
      writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle 
      declaredTypeHandle) at 
      WriteNodeTypeDtoToXml(XmlWriterDelegator , Object , 
      XmlObjectSerializerWriteContext , ClassDataContract) 
      at 
      System.Runtime.Serialization.ClassDataContract.WriteXmlValue(XmlWriterDelegator 
      xmlWriter, Object obj, XmlObjectSerializerWriteContext 
      context) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(XmlWriterDelegator 
      xmlWriter, Object obj, Boolean isDeclaredType, Boolean 
      writeXsiType, Int32 declaredTypeID, RuntimeTypeHandle 
      declaredTypeHandle) at 
      WriteArrayOfNodeTypeDtoToXml(XmlWriterDelegator , 
      Object , XmlObjectSerializerWriteContext , 
      CollectionDataContract) at 
      System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(XmlWriterDelegator 
      xmlWriter, Object obj, XmlObjectSerializerWriteContext 
      context) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.WriteDataContractValue(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithoutXsiType(DataContract 
      dataContract, XmlWriterDelegator xmlWriter, Object obj, 
      RuntimeTypeHandle declaredTypeHandle) at 
      System.Runtime.Serialization.DataContractSerializer.InternalWriteObjectContent(XmlWriterDelegator 
      writer, Object graph) at 
      System.Runtime.Serialization.DataContractSerializer.InternalWriteObject(XmlWriterDelegator 
      writer, Object graph) at 
      System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions(XmlWriterDelegator 
      writer, Object graph) at 
      System.Runtime.Serialization.XmlObjectSerializer.WriteObject(XmlDictionaryWriter 
      writer, Object graph) at 
      System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.SerializeParameterPart(XmlDictionaryWriter 
      writer, PartInfo part, Object graph) at 
      System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.SerializeParameter(XmlDictionaryWriter 
      writer, PartInfo part, Object graph) at 
      System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.SerializeBody(XmlDictionaryWriter 
      writer, MessageVersion version, String action, 
      MessageDescription messageDescription, Object 
      returnValue, Object[] parameters, Boolean isRequest) at 
      System.ServiceModel.Dispatcher.OperationFormatter.SerializeBodyContents(XmlDictionaryWriter 
      writer, MessageVersion version, Object[] parameters, 
      Object returnValue, Boolean isRequest) at 
      System.ServiceModel.Dispatcher.OperationFormatter.OperationFormatterMessage.OperationFormatterBodyWriter.OnWriteBodyContents(XmlDictionaryWriter 
      writer) at 
      System.ServiceModel.Channels.BodyWriter.WriteBodyContents(XmlDictionaryWriter 
      writer) at 
      System.ServiceModel.Channels.BodyWriterMessage.OnWriteBodyContents(XmlDictionaryWriter 
      writer) at 
      System.ServiceModel.Channels.Message.WriteBodyContents(XmlDictionaryWriter 
      writer) at 
      System.ServiceModel.Security.SecurityAppliedMessage.WriteBodyToSignThenEncryptWithFragments(Stream 
      stream, Boolean includeComments, String[] 
      inclusivePrefixes, EncryptedData encryptedData, 
      SymmetricAlgorithm algorithm, XmlDictionaryWriter 
      writer) at 
      System.ServiceModel.Security.WSSecurityOneDotZeroSendSecurityHeader.ApplyBodySecurity(XmlDictionaryWriter 
      writer, IPrefixGenerator prefixGenerator) at 
      System.ServiceModel.Security.SecurityAppliedMessage.OnWriteMessage(XmlDictionaryWriter 
      writer) at 
      System.ServiceModel.Channels.Message.WriteMessage(XmlDictionaryWriter 
      writer) at 
      System.ServiceModel.Channels.BufferedMessageWriter.WriteMessage(Message 
      message, BufferManager bufferManager, Int32 
      initialOffset, Int32 maxSizeQuota) at 
      System.ServiceModel.Channels.TextMessageEncoderFactory.TextMessageEncoder.WriteMessage(Message 
      message, Int32 maxMessageSize, BufferManager 
      bufferManager, Int32 messageOffset) at 
      System.ServiceModel.Channels.HttpOutput.SerializeBufferedMessage(Message 
      message) at 
      System.ServiceModel.Channels.HttpOutput.Send(TimeSpan 
      timeout) at 
      System.ServiceModel.Channels.HttpRequestContext.OnReply(Message 
      message, TimeSpan timeout) at 
      System.ServiceModel.Activation.HostedHttpContext.OnReply(Message 
      message, TimeSpan timeout) at 
      System.ServiceModel.Channels.RequestContextBase.Reply(Message 
      message, TimeSpan timeout) at 
      System.ServiceModel.Security.SecuritySessionServerSettings.SecuritySessionRequestContext.OnReply(Message 
      message, TimeSpan timeout) at 
      System.ServiceModel.Channels.RequestContextBase.Reply(Message 
      message, TimeSpan timeout) at 
      System.ServiceModel.Channels.RequestContextBase.Reply(Message 
      message) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Reply(MessageRpc&amp; 
      rpc) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessageCleanup(MessageRpc&amp; 
      rpc) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc&amp; 
      rpc) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc&amp; 
      rpc) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc&amp; 
      rpc) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc&amp; 
      rpc) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc&amp; 
      rpc) at 
      System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean 
      isOperationContextSet) at 
      System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Dispatch(MessageRpc&amp; 
      rpc, Boolean isOperationContextSet) at 
      System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext 
      request, Boolean cleanThread, OperationContext 
      currentOperationContext) at 
      System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext 
      request, OperationContext currentOperationContext) at 
      System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult 
      result) at 
      System.ServiceModel.Dispatcher.ChannelHandler.OnContinueAsyncReceive(Object 
      state) at 
      System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke2() 
      at 
      System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.OnSecurityContextCallback(Object 
      o) at 
      System.Security.SecurityContext.Run(SecurityContext 
      securityContext, ContextCallback callback, Object 
      state) at 
      System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.WorkItem.Invoke() 
      at 
      System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ProcessCallbacks() 
      at 
      System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.CompletionCallback(Object 
      state) at 
      System.ServiceModel.Channels.IOThreadScheduler.CriticalHelper.ScheduledOverlapped.IOCallback(UInt32 
      errorCode, UInt32 numBytes, NativeOverlapped* 
      nativeOverlapped) at 
      System.ServiceModel.Diagnostics.Utility.IOCompletionThunk.UnhandledExceptionFrame(UInt32 
      error, UInt32 bytesRead, NativeOverlapped* 
      nativeOverlapped) at 
      System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 
      errorCode, UInt32 numBytes, NativeOverlapped* 
      pOVERLAP)</StackTrace> 
      <ExceptionString>System.InsufficientMemoryException: 
      Failed to allocate a managed memory buffer of 536870912 
      bytes. The amount of available memory may be low. 
      ---&gt; System.OutOfMemoryException: Exception of type 
      'System.OutOfMemoryException' was thrown. at 
      System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32 
      size) --- End of inner exception stack trace 
      ---</ExceptionString> 
      <InnerException> 
       <ExceptionType>System.OutOfMemoryException, mscorlib, 
       Version=2.0.0.0, Culture=neutral, 
       PublicKeyToken=b77a5c561934e089</ExceptionType> 
       <Message>Exception of type 
       'System.OutOfMemoryException' was thrown.</Message> 
       <StackTrace>at 
       System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32 
       size)</StackTrace> 
       <ExceptionString>System.OutOfMemoryException: 
       Exception of type 'System.OutOfMemoryException' was 
       thrown. at 
       System.ServiceModel.Diagnostics.Utility.AllocateByteArray(Int32 
       size)</ExceptionString> 
      </InnerException> 
      </Exception> 
     </TraceRecord> 
     </DataItem> 
    </TraceData> 
    </ApplicationData> 
</E2ETraceEvent> 

Oczekiwanie na Nabeel

Answer 1

Czy na pewno macie bajt tylko około 2 MB? Z opublikowanego komunikatu o wyjątku wynika, że ​​komunikat w buforze zbliża się do 0,5 GB (536870912 bajtów). Czy ten błąd występuje konsekwentnie? Czy w tym czasie przeprowadzasz wiele zgłoszeń serwisowych?

Należy również pamiętać, że podczas korzystania z WSHttpBinding z Message Security cała pamięć jest buforowana. Rozumiem, że robi się to, aby zaszyfrować całą wiadomość w jednym bloku, aby odpowiednie informacje mogły zostać dostarczone w nagłówku. Innymi słowy, nagłówek potrzebuje długości wiadomości, która jest nieznana, dopóki szyfrowanie nie zostanie wykonane.

Zobacz moje powiązane pytanie, gdzie pytam How to best transfer large payloads of data using wsHttp with WCF with message security.

Answer 2

Nasza pula aplikacji została ustawiona na Odzyskiwanie co 0 minut (bez recyklingu). Po zmianie z powrotem na 1740 i ręcznym recyklingu puli błąd zniknął. Myślę, że Pula aplikacji kończy się dostępną/zatwierdzoną pamięcią i zwraca ten komunikat, jeśli wszystkie twoje obiekty nie zostały prawidłowo umieszczone w .NET

Answer 3

Mieliśmy również dokładnie ten sam problem. Próbowałem pobrać dokument o wielkości zaledwie 15 MB, a niebo wykorzystania pamięci przyspieszyło do około 1 GB. Spowodowało to wyjątek wyjątku na serwerze. Rozwiązanie dla nas było bardzo proste. Umowa danych wdrożyła ICollection dla ByteArray. Kiedy zmieniliśmy to na Bajt [], problem zniknął, a zużycie pamięci znów było normalne. Wygląda na to, że proces serializacji nie może obsłużyć typu ICollection w efektywny sposób.