如何用IStream传递一个较大的文件？ 如何Mashaling该stream？

[romijn]

在客户和DCom服务器中间传送文件，以前我是用VARIANT包装了SAFEARRAY来传递数据的，但是对于大文件13M左右，读很快几秒钟即可，但是写很慢，要1分钟左右。
ReadData([out] VARIANT *pBuf);//从DCom读数据，传到前端，很快13M要几秒钟
WriteData([in]VARIANT *pBuf);//前端上传数据到DCom，很慢要13M要1分钟
不知道是为什么，同样的要传递13M，只是[in]、[out]的区别，发现WriteData大部分的时间都耗费在参数的传递上了。
经网友的提示到国外网站上看看，发现很多人都遇到了这个问题，都说要用IStream传递。但是要把IStream列集和散集才行，我不知道怎么做？有人试过吗？
DCom服务器利用IStream的函数如下：
WriteStreamFile1(IStream *pITransferStream, BSTR psFileName, BSTR psCheckStr, int piSize)；
利用IStream的几个函数（stat和read）做测试，发现IStream流不能够把数据传送过去，据说是要列集和散集，有人做过吗？在客户端，已经测试过IStream的读写，都没有问题。
困惑中......

[romijn]

up

[xystarch]

看看这个对你有没有帮助

// Your CSimpleObj class looks like this:
class CSimpleObj : public CObject
{
DECLARE_SERIAL( CSimpleObj )
public:
// constructor and destructor
CSimpleObj();
virtual ~CSimpleObj();
// Set internal string data
void SetString( CString csData );
// Used to serialize data into an archive
virtual void Serialize(CArchive& ar);
// Display the data string
void Show();
private:
CString m_strData;// Internal data string
};

// Write this object to an archive
void CSimpleObj::Serialize(CArchive& ar)
{
CObject::Serialize( ar );

if (ar.IsLoading())
{
// extract data from archive
ar >> m_strData;
}
else
{
// store data into archive
ar << m_strData;
}
}

// Method to display data in this object
void CSimpleObj::Show()
{
AfxMessageBox(m_strData);
}

// save a string in data member
void CSimpleObj::SetString(CString csData)
{
m_strData = csData;
}

Now, the next step is Serializing and De-Serializing (Loading and Storing Objects) with a CArchive. I'm using a different class called CBlob for it.

class CBlob
{
public:
CBlob() {};
virtual ~CBlob() {};

// Extract data from a CObject and load it into a SAFEARRAY.
SAFEARRAY* Load( CObject *pObj );
// Re-create an object from a SAFEARRAY
BOOL Expand( CObject * &pObj, SAFEARRAY *pVar );
private:
};

// Extract data from a CObject and use it to create a SAFEARRAY.
SAFEARRAY* CBlob::Load( CObject *pObj)
{
CMemFile memfile; // memory file

// define the flag that tells the archive whether it should
// load or store
long lMode = CArchive::store | CArchive::bNoFlushOnDelete;
// create the archive using the memory file
CArchive ar(&memfile, lMode );

// m_pDocument is not used
ar.m_pDocument = NULL;
// serialize the object into the archive
ar.WriteObject(pObj);
// close the archive -- the data is now stored in memfile
ar.Close();

// get the length (in bytes) of the memory file
long llen = memfile.GetLength();

// detach the buffer and close the file
unsigned char *pMemData = memfile.Detach();

// set up safearray
SAFEARRAY *psa;

// create a safe array to store the stream data
psa = SafeArrayCreateVector( VT_UI1, 0, llen );

// pointers to byte arrays
unsigned char *pData = NULL;

// get a pointer to the safe array. Locks the array.
SafeArrayAccessData( psa, (void**)&pData );

// copy the memory file into the safearray
memcpy( pData, pMemData, llen );

// clean up buffer
delete pMemData;

// unlock access to safearray
SafeArrayUnaccessData(psa);

// return a pointer to a SAFEARRAY allocated here
return psa;
}

// Re-create an object from a SAFEARRAY
BOOL CBlob::Expand(CObject * &rpObj, SAFEARRAY *psa)
{
CMemFile memfile; // memory file for de-serailze
long lLength; // number of bytes
char *pBuffer; // buffer pointer

// lock access to array data
SafeArrayAccessData( psa, (void**)&pBuffer );

// get number of elements in array. This is the number of bytes
lLength = psa->rgsabound->cElements;

// attach the buffer to the memory file
memfile.Attach((unsigned char*)pBuffer, lLength);

// start at beginning of buffer
memfile.SeekToBegin();

// create an archive with the attached memory file
CArchive ar(&memfile, CArchive::load | CArchive::bNoFlushOnDelete);

// document pointer is not used
ar.m_pDocument = NULL;

// inflate the object and get the pointer
rpObj = ar.ReadObject(0);

// close the archive
ar.Close();

// Note: pBuffer is freed when the SAFEARRAY is destroyed
// Detach the buffer and close the file
pBuffer = (char*) memfile.Detach();

// release the safearray buffer
SafeArrayUnaccessData( psa );

return TRUE;
}

[IAnonymous]

我试验了你说的情况，但是好像没有你说的差别这么大，

我用的48.9MB的MTV来做 ReadData 和 WriteData 都是10s左右，

也就是说和你直接在两台机器上拷贝粘贴速度一样。

方法都是Buffer->SafeArray->VARIANT。

测试环境百兆以太网 ， 两台P4的机器.

[harry202]

介绍一下Mashaling如何？一直不太明白，甚至没看到过Mashaling在中文叫什么。。郁闷。

[IAnonymous]

好像叫做列集 散列 什么的，总之是RPC调用过程中的一种参数调整吧，
具体好想是由存根和代理dll实现的，总之我也不清楚，编程序不用关心太多细节

[romijn]

up.等待中

[zzyx]

以下是我使用IStream作为参数写入COM组件的测试结果
Write byte: 10950000
Call raw_WriteData2Stream cost time is 13 second
Innner raw_WriteData2Stream cost time is:13

接口方法 WriteData2Stream(IStream* pStream,long nSize)
方法实现中：从IStream读取nSize指定的数据，然后写入文件（以检查是否真正传递了数据）

组件在COM+中注册
导出代理到另外一个计算机，在另外的计算机上运行客户端的测试程序，得到结果。

我的结果看起来也有点奇怪：
第一个13秒，是WriteData2Stream 整个执行耗费的时间
第二个13秒，WriteData2Stream 内部一开始执行就记录时间，完成从Stream中读取，写入文件，到返回之前耗费的时间。

看这个结果，似乎传输只花费了0或不到1秒钟的时间？

使用IStream方法的问题是肯定不会支持脚本。

[zzyx]

从资料上看，好像IStream不需要自己进行marshaling。

引用：Marshaling Your Data: Efficient Data Transfer Techniques Using COM and Windows 2000

The final method of transferring data that I want to mention is the use of stream objects. IStream pointers can be marshaled by type library marshaling, and can be accessed by C++ clients.

[romijn]

菜农同志，能否讲的详细一点？com＋注册和dcom的注册有何区别？

[zzyx]

//数据写入文件中
STDMETHODIMP CComA::WriteData2Stream(IStream* pStream, int nSize,BSTR* strRet)
{
// TODO: Add your implementation code here
DWORD dwBegin=GetTickCount();


pStream->AddRef ();
CComPtr<IStream> p(pStream);
HRESULT hr;

LARGE_INTEGER x;
x.QuadPart =0;

hr=p->Seek(x,STREAM_SEEK_SET,0 );

int hFile=_open("c:\\xml\\com.out.dat",_O_CREAT|_O_RDWR,_S_IWRITE);

BYTE* buf=new BYTE[nSize];
DWORD nRead;
p->Read (buf,nSize,&nRead);
if(hFile>0)
{
write(hFile,buf,nRead);
_close(hFile);
}

delete buf;

pStream->Release();

DWORD dwEnd=GetTickCount();
char time[32];
memset(time,0,32);
ltoa((dwEnd-dwBegin)/1000,time,10);
CComBSTR t="Run Time Length:";
t+=time;
*strRet=SysAllocString(t.Copy());
return S_OK;

}

//客户端调用
void test1(int nCnt)
{
IComAPtr p=NULL;
HRESULT hr;

hr=p.CreateInstance ("MyCom.ComA");

if(FAILED(hr))
{
printf("\nError Create ComA:%d",hr);
return;
}
printf("\nBefor Create a IStream");

CComPtr<IStream> pStream=NULL;

long nSize=1024;
HGLOBAL h=GlobalAlloc(GMEM_MOVEABLE,nSize);
LPVOID pv=GlobalLock(h);
hr=CreateStreamOnHGlobal(h,false,&pStream);
try{
//写入数据到stream
char buf[1024];
DWORD nRet;
memset(buf,0,1024);
strcat(buf,"1AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\r\n");
strcat(buf,"2BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\r\n");
strcat(buf,"3CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC\r\n");
strcat(buf,"4DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD\r\n");
strcat(buf,"5EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE\n\r");
strcat(buf,"6FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF\r\n");
strcat(buf,"7GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG\r\n");
strcat(buf,"8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\r\n");
strcat(buf,"9BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\r\n");
strcat(buf,"0CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC\r\n");
strcat(buf,"1AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\r\n");
strcat(buf,"2BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\r\n");
strcat(buf,"3CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC\r\n");
strcat(buf,"4DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD\r\n");
strcat(buf,"5EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE\n\r");
strcat(buf,"6FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF\r\n");
strcat(buf,"7GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG\r\n");
strcat(buf,"8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\r\n");
strcat(buf,"9BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\r\n");
strcat(buf,"0CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC\r\n");
strcat(buf,"end\r\n");

nSize=0;
for(int i=0;i<nCnt;i++)
{
pStream->Write (buf,1024,&nRet);

nSize+=nRet;
printf("\nWrite Stream byte: %d",nSize);
}
//调用
DWORD dwBegin=GetTickCount();
BSTR strRet=NULL;
hr=p->raw_WriteData2Stream (pStream,nSize,&strRet);
DWORD dwEnd=GetTickCount();
printf("\nCall raw_WriteData2Stream cost time is %d second",(dwEnd-dwBegin)/1000);
_bstr_t t(strRet);
printf("\nInnner raw_WriteData2Stream cost time is:%s",(char*)t);

}
catch(...)
{
printf("\nCall WriteData2Stream Error");
}

GlobalUnlock(h);
GlobalFree(h);


}

[zzyx]

com＋注册和dcom的注册有何区别？

我也没有办法深入理解。我的理解，COM+的底层在分布式这点上来说，和DCOM没有区别，只是管理起来更方便一些。

应用上，在2k中的“组件服务”，可以在A的com+中，为组件X创建一个应用
然后利用其工具可以把X导出代理为x，
在B计算机上安装x即可。

这样，B计算机上的客户端程序，在创建X的实例的时候，实际上通过x调用了A上的X
如果用DCOM，代理，存根之类的事情，让人头疼，事实上，我现在也没有太明白直接注册代理存根到底是怎么回事情。

呵呵，我理解应该和DCOM没有什么区别。

[zzyx]

另外，我的以上代码是测试使用，中间好像有bug。
再一个问题，在客户端导入组件（import）的时候，有一些warning比较讨厌，还不知道该怎么处理。似乎和组件的自动化接口有关系。
我想，如果把IStream参数声明为IUnkown，在客户端和服务器端进行转换，应该可以避免。懒得试了。

[zzyx]

但，使用IUnkonw，调度程序是否还有效工作，就又是一个问题了。

[romijn]

谢谢菜农，我会尽快测试！！把结果反馈上来！！
我的服务器段的代码除了这两行，基本上和你是一样的。我是直接用传过来的参数pStream来读的，根据pStream可以知道流的大小。但是我的只能知道大小，读不出数据。
pStream->AddRef ();
ComPtr<IStream> p(pStream);
另外一点：我的客户段和服务器是不同的机器，采用dcom方式调用。
我的服务端在服务器和客户机两台机器上均采用 server.exe /regserver注册
根据你的代码hr=p.CreateInstance ("MyCom.ComA");
，你的是在同一台机器上运行的吧，那样的话速度又是另外一回事了！

服务端：
STDMETHODIMP CComInter::WriteStreamFile1(IStream *pITransferStream, BSTR psFileName, BSTR psCheckStr, int piSize)
{
long iTime=GetTickCount();
//打开日志！
FILE *fout;
BOOL b=FALSE;
if( (fout = fopen( pProDir, "a+t" )) != NULL ){
fprintf(fout,"\n /*****************************************");
fprintf(fout,"\n FILE:%s",pPathBuf);
b=TRUE;
}

HANDLE hClient = CreateFile("d:\temp\abc.twd",
GENERIC_WRITE |GENERIC_READ,0,NULL,CREATE_ALWAYS,FILE_ATTRIBUTE_NORMAL,NULL);

if(hClient != INVALID_HANDLE_VALUE)
ULARGE_INTEGER cbSeek;
LARGE_INTEGER cbInit;
cbInit.QuadPart=0;
pITransferStream->Seek(cbInit,STREAM_SEEK_SET,&cbSeek);
BYTE *pByte;
pByte=new BYTE[piSize];
ULONG uLong1,uLong2;
uLong1=piSize;
pITransferStream->Read(pByte,uLong1,&uLong2);
DWORD dwWritten;
long iT1=GetTickCount();
::WriteFile(hClient,pByte,piSize,&dwWritten,NULL);
CloseHandle(hClient);//close pipe
long iT2=GetTickCount()-iT1;
if(b){
fprintf(fout,"\n 使用了APIwrite参数！");
fprintf(fout," \n Apiwrite函数共耗时：%li毫秒",iT2);
fprintf(fout,"\n 文件的长度为:%li",piSize);
fprintf(fout," \n Use c sussess!!!");
}
}
long iTime2=GetTickCount()-iTime;
if(b){
fprintf(fout," \n 执行写文件com函数共耗时：%li毫秒",iTime2);
fclose(fout);
}
return S_OK;
}
客户端连接dcom方式：
IComInter1 = NULL;
srvinfo.dwReserved1=0;
//srvinfo.pwszName=L"192.168.0.2";
srvinfo.pwszName=L"192.168.0.2";
srvinfo.pAuthInfo=NULL;
srvinfo.dwReserved2=0;
MULTI_QI mqi;
mqi.pIID=&IID_IComInter;
mqi.pItf=NULL;
mqi.hr=0;
HRESULT hr;
hr = CoInitialize(0);
if(SUCCEEDED(hr))
{
hr = CoCreateInstanceEx( CLSID_ComInter, NULL, CLSCTX_SERVER,&srvinfo,1, &mqi);
if (SUCCEEDED(hr))
{
hr = mqi.pItf->QueryInterface(IID_IComInter,(void**) &IComInter1);
return true;
}
}

[zzyx]

我的测试过程也是分布在两个计算机上的。

hr=p.CreateInstance ("MyCom.ComA");
是因为使用了COM+导出的代理，比较方便。
组件实际运行在另外一台计算机上的，客户端程序所在的只有一个代理而已。

你的代码实现的方式是用客户端程序指定计算机，
而我的代码实现是通过COM+自动指定了计算机
最多在创建实例的时候可能内部流程有些差异，但调用方法的过程应该一致的，也就是Marshal应该一样的。

难道COM+和DCOM真的有不同？

至于
pStream->AddRef ();
ComPtr<IStream> p(pStream);
应该是不对。AddRef没有必要，因为CComPtr自动完成这个工作。后边也不用调用Release。

另外，你的程序是exe执行文件吗？
也许这是我们之间的区别？
COM+不能调度exe，只能调度dll
独立的exe的调度，是自己管理组件的，那么也许调度上就有麻烦了。

建议你试验DLL类型看看。当然，dll类型在DCOM的配置上比较麻烦，所以可以使用COM+管理器完成。

[romijn]

菜农同志，这两天加班赶任务，谢谢你！！
过两天再给你答复！！

[romijn]

菜农同志，我这边还是有问题，用IStream传递数据时，编译出现一个警告，如下：
E:\vc project\dcom\DataSvr\DataSvr.idl(21) : warning MIDL2039 : interface does not conform to [oleautomation] attribute : [ Parameter 'pITransferStream' of Procedure 'WriteStreamFile' ( Interface 'IDcominter' ) ]
我的数据写不到服务器上。根据提示，是因为IStream不是oleautomation的类型，所以需要创建一个dll，用Nmake创建了datasvrps.dll，在服务器和客户机上注册该dll。
运行我的程序，发现从istream中无法读出数据。我的服务端的代码如下：
STDMETHODIMP CDcominter::WriteStreamFile(IStream *pITransferStream, BSTR psFileName, BSTR psCheckStr, int piSize)
{

long iTime=GetTickCount();

SYSTEMTIME systime;
GetLocalTime(&systime);
//打开日志！
FILE *fout;
BOOL b=FALSE;
if( (fout = fopen("E:\\Ghy_Share\\DCOM\\LOG.TXT", "a+t" )) != NULL ){
fprintf(fout,"\n /**************************************************");
fprintf(fout,"\n DateTime: %4i/%2i/%2i %2i:%2i:%2i",
systime.wYear,systime.wMonth,systime.wDay,systime.wHour,
systime.wMinute,systime.wSecond);
//fprintf(fout,"\n filename:%s",aFileName);
b=TRUE;
}


HANDLE hClient = CreateFile("E:\\Ghy_Share\\DCOM\\abc.twd",
GENERIC_WRITE |GENERIC_READ,0,NULL,CREATE_ALWAYS,FILE_ATTRIBUTE_NORMAL,NULL);


if(hClient == INVALID_HANDLE_VALUE)
{
fprintf(fout," Api failed!!!");
}
else
{
pITransferStream->AddRef ();
CComPtr<IStream> pStream(pITransferStream);


ULARGE_INTEGER cbSeek;
LARGE_INTEGER cbInit;
cbInit.QuadPart=0;
pStream->Seek(cbInit,STREAM_SEEK_SET,&cbSeek); // reset stream to its 0 offset

BYTE *pByte;
pByte=new BYTE[piSize];
ULONG uLong1,uLong2;
uLong1=piSize;
pStream->Read(pByte,uLong1,&uLong2);


DWORD dwWritten;

long iT1=GetTickCount();
//::WriteFile(hClient,pDest,pBuf->parray->rgsabound->cElements,&dwWritten,NULL);
WriteFile(hClient,pByte,piSize,&dwWritten,NULL);
CloseHandle(hClient);//close pipe
long iT2=GetTickCount()-iT1;
if(b){
fprintf(fout,"\n 使用了APIwrite参数！");
fprintf(fout," \n Apiwrite函数共耗时：%li毫秒",iT2);
fprintf(fout," \n 从流中读取字节为：%li",uLong2);
fprintf(fout,"\n 文件的长度为:%li",piSize);
fprintf(fout," \n Use c sussess!!!");
}

}


long iTime2=GetTickCount()-iTime;
if(b){
fprintf(fout," \n 执行写文件com函数共耗时：%li毫秒",iTime2);
fclose(fout);
}
return S_OK;
｝
日志如下：

/**************************************************
DateTime: 2003/ 4/23 14:59:15
使用了APIwrite参数！
Apiwrite函数共耗时：0毫秒
从流中读取字节为：0
文件的长度为:705
Use c sussess!!!
执行写文件com函数共耗时：63毫秒

[zzyx]

有个问题：
你使用的是exe类型的com还是dll类型的com？这涉及到由谁来调度的问题。我的测试是dll，注册在COM+中，由COM+来调度的。

我现在没有合适的环境去测试，等有机会我再测试一下使用exe，在dcom中完成配置的情况。

[romijn]

我的是Exe程序！！你在编译中没有出现类似的警告吗？E:\vc project\dcom\DataSvr\DataSvr.idl(21) : warning MIDL2039 : interface does not conform to [oleautomation] attribute : [ Parameter 'pITransferStream' of Procedure 'WriteStreamFile' ( Interface 'IDcominter' ) ]
说是IStream不是自动化类型的变量？

[romijn]

在某个外国的论坛上面看到也有关这个问题的描述，上面说这条警告只是编译器的一个bug，但是我的问题依然存在！！困惑！http://www.grimes.demon.co.uk/DCOM/DCOM/Interfaces.htm


Question
From: Brian Muth <[e-mail]>
Date: 98/12/29 13:37
Subject: Marshalling IStream through standard marshaller

I'm interested in using a method that avoids an extra roundtrip by returning a pointer to IStream rather than IUnknown (then QI'ing for IStream). The method's interface must be of type dual, but I get the warning message "warning MIDL2039 : interface does not conform to [oleautomation]...".

My neural synapses fail to fire when reviewing the archives. Can someone explain the problem to me (and suggest a workaround?).

Response
From: Ronald Laeremans <[e-mail]>
Date: 98/12/29
Subject: Re: marshalling IStream through standard marshaller

I am almost 100% certain that the Universal Marshaller will correctly marshal this (it simply has no reason why it shouldn't) and that the warning is just bogus. I can only guess that the developer who put the warning in wants to warn you that if you subsequently use the IStream, method calls on that interface will not be marshallable by the Universal Marshaller. It is beyond me why one would care.

Sadly, I haven't gotten a confirmation on this from MS although I have tried in earnest several times.

Response
From: Michael Nelson <[e-mail]>
Date: 98/12/30
Subject: Re: marshalling IStream through standard marshaller

The universal marshaler does not need to know how to marshal IStream, a proxy/stub already exists for it on the system. The fact that you're passing an IStream through a function in your unimarshaled interface is irrelevant.

All specifying IStream as the parameter type does is let the marshaler know the IID to QI() when objects go through it.

The error message is not there to say it isn't going to let you pass IStream, it's there to let you know that you interface may be incompatible with pure automation clients which may try to use that function.

-mike