谁实现了java rmi 的压缩

[lujianfu]

如题，我依据下面文章来的，但是不成功，不知道哪个大神实现了。



第一个问题是: 我怎样才能压缩那些不在文件中的数据.

第二个问题是: 我以极大的热情阅读了Todd Sundsted的"压缩你的数据,从而提高你的网络应用程序的性能",但是读完后我却有点失望.当我读到文章标题时我很高兴.我想我总算找到了解决问题的办法了.

在我们的公司,我们试图提高一个组织数据的RMI应用程序的性能.服务器端进行了绝大部分的处理和优化.我们花了一年半的时间去提高性能,但是现在看来瓶颈在于数据的传输上.在一天的任何时间内,我们都有可能在客户和服务器之间传送成千上万的数据.

一种可能的解决办法,我建议我们能够在把数据返回给客户端时先压缩这些数据,这在Todd的文章中已经说得很清楚了.但是,文章中的例子却是压缩文件,而不是我们所需要的----对数据进行压缩.

在RMI中的实现中,我们先从数据库取得数据,再把数据放入一个列表中,接着把这个列表返回给客户端,最后再把它们插入JTable中.我想在把数据返回给客户时,首先把列表中的数据压缩,然后在客户端解压缩,最后把数据插入到表格中.

这样的想法可行吗?

A:最近我收到了一些关于Todd的文章的疑问.很多读者看起来对文章中的举例很疑惑.因为文章中的例子是以文件压缩为核心的.

首先回答第一个问题,当你使用ZipInputStream 和 ZipOutputStream 并没有强制你必须使用文件.唯一要注意的是你必须把数据转换为字节数组的形式.

第二个问题比较棘手.在网络中,以RMI方式通信就需要作一些调整了.为了在传送数据之前就让RMI进行数据压缩,你必须创建一个能够压缩数据的新的套接字.然后,当你创建了一个套接字后,你得告诉RMI使用这一套接字.

以下是创建一个RMI形式的套接字的简要步骤:

1:选择或者创建一个新的套接字.(可以参看SUNS的"创建一个典型的套接字").

2:创建一个服务器端的套接字.

3:创建一个RMIClientSocketFactory

4:创建一个RMIServerSocketFactory

5:创建一个继承了UnicastRemoteObjec的远程对象,从而使用新的factories.

根据这一大致的想法,我们来看每一步如何具体的实现.

步骤1: 创建ZipSocket

由于要进行Zip压缩,我们重新创建这样的套接字

mport java.io.InputStream;

import java.io.OutputStream;

import java.util.zip.ZipInputStream;

import java.util.zip.ZipOutputStream;

import java.net.Socket;

public class ZipSocket extends Socket {



private InputStream in;

private OutputStream out;



public ZipSocket() { super(); }



public ZipSocket(String host, int port)

throws IOException {

super(host, port);

}



public InputStream getInputStream()

throws IOException {

if (in == null) {

in = new ZipInputStream(super.getInputStream());

}

return in;

}



public OutputStream getOutputStream()

throws IOException {

if (out == null) {

out = new ZipOutputStream(super.getOutputStream());

}

return out;

}



public synchronized void close() throws IOException {

OutputStream o = getOutputStream();

o.flush();

super.close();

}

}





步骤2: 创建ZipServerSocket



import java.net.ServerSocket;

import java.net.Socket;

import java.io.IOException;



public class ZipServerSocket extends ServerSocket

{

public ZipServerSocket(int port) throws IOException {

super(port);

}



public Socket accept() throws IOException {

Socket socket = new ZipSocket();

implAccept(socket);

return socket;

}

}





步骤3:创建ZipClientSocketFactory

客户端的factory的创建必须遵循以下的形式:



import java.io.IOException;

import java.io.Serializable;

import java.net.Socket;

import java.rmi.server.RMIClientSocketFactory;



public class ZipClientSocketFactory

implements RMIClientSocketFactory, Serializable {

public Socket createSocket(String host, int port)

throws IOException {

ZipSocket socket = new ZipSocket(host, port);

return socket;

}

}



步骤4:创建ZipServerSocketFactory



import java.io.IOException;

import java.io.Serializable;

import java.net.ServerSocket;

import java.rmi.server.RMIServerSocketFactory;



public class ZipServerSocketFactory

implements RMIServerSocketFactory, Serializable {



public ServerSocket createServerSocket(int port)

throws IOException {

ZipServerSocket server = new ZipServerSocket(port);

return server;

}

}



步骤5: 创建一个继承了UnicastRemoteObjec的远程对象,从而使用新的factories.

public class YourRMIObject extends UnicastRemoteObject {



public YourRemoteObject( int port ) {

super( port, new ZipClientSocketFactory(), new ZipServerSocketFactory() );

}



// 剩下的是你自己的程序实现



}

[lujianfu]

希望能依据上面的文帐，写个切实可运行的demo

[lujianfu]

高手呢。。。。。

[惑惑]

文字太多，代码没格式，看得头晕

[龙四]

William Grosso的《JAVA RMI》一书中有例子

[lujianfu]

上面的方法我曾经试过，老是说流不能到结尾 无意中发现一个牛人冲写了这个压缩流，终于可以 package common.zip; import java.io.EOFException; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.util.logging.Level; import java.util.logging.Logger; /** * This code is based on the ideas presented in {@link http://javatechniques.com/blog/compressing-data-sent-over-a-socket/} by * Philip Isenhour. I am very grateful for the approach that he presented. The key idea is to avoid using the GZip and the Zip streams * and to use the Deflater and Inflater methods directly. In addition, Philip Isenhour essentially defines a packet type that has a * header indicating the size and compressed size of the packet contents. I took these two key ideas and wrote the following code * by scratch without reference to Philip Isenhour's documents. I think that some version of Philip Isenhour's ideas should find their way * into the core java libraries because otherwise people will continue struggling with this problem. * <p> * I have tried several other approaches to a compressing input and compressing output stream. The first approach was to base the * input and output streams on the GZip input and output stream. There are web pages on the internet that suggest that calling the GZipOutputStream's * finish() method during the flush() will work. I had trouble with this approach when a write occurs on the stream after * the flush() (which calls finish()). I would get exceptions indicating that the GZip Output stream was finished and therefore unwriteable. * <p> * I then tried to use the ZIPInput/OutputStreams. I would flush data by creating a ZipEntry and writing it out. This approach * actually worked very well. But it had a mysterious bug where some data was either not fully written out or not read. In the rmi * context things would hang. This bug was relatively rare and only happened on certain machines. I never found out what the problem was. * <p> * The beauty of Philip Isenhour's approach is that the developer can completely control how data is flushed and fully written out. The developer * can also ensure that on the read method all the data is fully read. So there should not be any more rmi hangs. The only issue is whether the * deflate/inflate logic is correct. This is pretty thoroughly tested in our server-client testing (though there are *always* bugs hidden somewhere). * * @author tredmond * */ public class PacketHeader { public static byte [] ALIGNMENT = { 0x4c, 0x3a, 0x74, 0x58 }; private static int BYTES_IN_INT = 4; private static int BITS_IN_BYTE = 8; private static int BYTE_MASK = 0x0ff; private int size; private int compressedSize; public PacketHeader(int size, int compressedSize) { this.size = size; this.compressedSize = compressedSize; } public static PacketHeader read(InputStream is) throws IOException { for (int i=0;i<ALIGNMENT.length;i++) { byte b=ALIGNMENT[i]; int alignCheck = is.read(); if (alignCheck == -1) { throw new EOFException("No packet found"); } if ((byte) alignCheck != b) { throw new IOException("Packet header out of alignment between reader and writer (Thread = " + Thread.currentThread().getName() + ")"); } } int size = readInt(is); int compressedSize = readInt(is); return new PacketHeader(size, compressedSize); } public void write(OutputStream os) throws IOException { for (int i=0;i<ALIGNMENT.length;i++) { byte b=ALIGNMENT[i]; os.write(b); } writeInt(os, size); writeInt(os, compressedSize); } public int getSize() { return size; } public int getCompressedSize() { return compressedSize; } private static int readInt(InputStream is) throws IOException { int result = 0; int[] buffer = new int[BYTES_IN_INT]; for (int i = 0; i < BYTES_IN_INT; i++) { int c = is.read(); if (c == -1) { throw new EOFException("Could not read compressed packet header"); } buffer[i] = c; } for (int i = BYTES_IN_INT - 1; i >= 0; i--) { result = result << BITS_IN_BYTE; int b = buffer[i]; result += b < 0 ? 256 + b : b; } return result; } private static void writeInt(OutputStream os, int v) throws IOException { for (int i = 0; i < BYTES_IN_INT - 1; i++) { os.write(v & BYTE_MASK); v = v >> BITS_IN_BYTE; } os.write(v); } }

[lujianfu]

package common.zip; import java.io.EOFException; import java.io.IOException; import java.io.InputStream; import java.util.logging.Level; import java.util.logging.Logger; import java.util.zip.DataFormatException; import java.util.zip.Inflater; /** * This code is based on the ideas presented in {@link http://javatechniques.com/blog/compressing-data-sent-over-a-socket/} by * Philip Isenhour. I am very grateful for the approach that he presented. The key idea is to avoid using the GZip and the Zip streams * and to use the Deflater and Inflater methods directly. In addition, Philip Isenhour essentially defines a packet type that has a * header indicating the size and compressed size of the packet contents. I took these two key ideas and wrote the following code * by scratch without reference to Philip Isenhour's documents. I think that some version of Philip Isenhour's ideas should find their way * into the core java libraries because otherwise people will continue struggling with this problem. * <p> * I have tried several other approaches to a compressing input and compressing output stream. The first approach was to base the * input and output streams on the GZip input and output stream. There are web pages on the internet that suggest that calling the GZipOutputStream's * finish() method during the flush() will work. I had trouble with this approach when a write occurs on the stream after * the flush() (which calls finish()). I would get exceptions indicating that the GZip Output stream was finished and therefore unwriteable. * <p> * I then tried to use the ZIPInput/OutputStreams. I would flush data by creating a ZipEntry and writing it out. This approach * actually worked very well. But it had a mysterious bug where some data was either not fully written out or not read. In the rmi * context things would hang. This bug was relatively rare and only happened on certain machines. I never found out what the problem was. * <p> * The beauty of Philip Isenhour's approach is that the developer can completely control how data is flushed and fully written out. The developer * can also ensure that on the read method all the data is fully read. So there should not be any more rmi hangs. The only issue is whether the * deflate/inflate logic is correct. This is pretty thoroughly tested in our server-client testing (though there are *always* bugs hidden somewhere). * * @author tredmond * */ public class CompressingInputStream extends InputStream { protected InputStream is; protected byte buffer[]; protected int offset; private Inflater inflater; private static int counter = 0; private int id; public CompressingInputStream(InputStream is) { this.is = is; inflater = new Inflater(); synchronized (CompressingInputStream.class) { id = counter++; } } public int read() throws IOException { if (buffer == null) { fillBuffer(); } if (buffer == null) { return -1; } int ret = buffer[offset++]; if (buffer.length == offset) { buffer = null; } return ret; } public int read(byte[] b, int off, int len) throws IOException { if (buffer == null) { fillBuffer(); } if (buffer == null) { return -1; } int bytesRead = 0; for (bytesRead = 0; offset < buffer.length && bytesRead < len; bytesRead++) { b[off++] = buffer[offset++]; } if (buffer.length == offset) { buffer = null; } return bytesRead; } private void fillBuffer() throws IOException { buffer = null; offset = 0; PacketHeader header = PacketHeader.read(is); buffer = new byte[header.getSize()]; fillBuffer(header); } protected void fillBuffer(PacketHeader header) throws IOException { inflater.reset(); int compressedSize = header.getCompressedSize(); byte compressedBuffer[] = new byte[compressedSize]; readFully(compressedBuffer, compressedSize); inflater.setInput(compressedBuffer); try { int inflatedSize = inflater.inflate(buffer); if (inflatedSize != header.getSize()) { throw new IOException("Inflated to the wrong size, expected " + header.getSize() + " bytes but got " + inflatedSize + " bytes"); } } catch (DataFormatException dfe) { IOException ioe = new IOException("Compressed Data format bad: " + dfe.getMessage()); ioe.initCause(dfe); throw ioe; } if (!inflater.needsInput()) { throw new IOException("Inflater thinks that there is more data to decompress"); } logPacket(compressedBuffer); } protected void readFully(byte[] b, int len) throws IOException { int bytesRead = 0; while (bytesRead < len) { int readThisTime = is.read(b, bytesRead, len - bytesRead); if (readThisTime == -1) { throw new EOFException("Unabled to read entire compressed packet contents"); } bytesRead += readThisTime; } } protected void logPacket(byte [] compressedBuffer) { try { StringBuffer sb = new StringBuffer(); sb.append("Uncompressed buffer of size "); sb.append(buffer.length); sb.append(": "); for (int i = 0; i < buffer.length; i++) { sb.append(buffer[i]); sb.append(" "); } sb = new StringBuffer(); sb.append("Compressed buffer of size "); sb.append(compressedBuffer.length); sb.append(": "); for (int i = 0; i < compressedBuffer.length; i++) { sb.append(compressedBuffer[i]); sb.append(" "); } } catch (Throwable t) { } } }

[lujianfu]

package common.zip; import java.io.IOException; import java.io.OutputStream; import java.util.zip.Deflater; /** * This code is based on the ideas presented in {@link http://javatechniques.com/blog/compressing-data-sent-over-a-socket/} by * Philip Isenhour. I am very grateful for the approach that he presented. The key idea is to avoid using the GZip and the Zip streams * and to use the Deflater and Inflater methods directly. In addition, Philip Isenhour essentially defines a packet type that has a * header indicating the size and compressed size of the packet contents. I took these two key ideas and wrote the following code * by scratch without reference to Philip Isenhour's documents. I think that some version of Philip Isenhour's ideas should find their way * into the core java libraries because otherwise people will continue struggling with this problem. * <p> * I have tried several other approaches to a compressing input and compressing output stream. The first approach was to base the * input and output streams on the GZip input and output stream. There are web pages on the internet that suggest that calling the GZipOutputStream's * finish() method during the flush() will work. I had trouble with this approach when a write occurs on the stream after * the flush() (which calls finish()). I would get exceptions indicating that the GZip Output stream was finished and therefore unwriteable. * <p> * I then tried to use the ZIPInput/OutputStreams. I would flush data by creating a ZipEntry and writing it out. This approach * actually worked very well. But it had a mysterious bug where some data was either not fully written out or not read. In the rmi * context things would hang. This bug was relatively rare and only happened on certain machines. I never found out what the problem was. * <p> * The beauty of Philip Isenhour's approach is that the developer can completely control how data is flushed and fully written out. The developer * can also ensure that on the read method all the data is fully read. So there should not be any more rmi hangs. The only issue is whether the * deflate/inflate logic is correct. This is pretty thoroughly tested in our server-client testing (though there are *always* bugs hidden somewhere). * * @author tredmond * */ public class CompressingOutputStream extends OutputStream { public static int COMPRESSION_PAD = 1024; public static int BUFFER_SIZE = 128 * 1024; public static int KB = 1024; protected OutputStream os; private Deflater deflater; private static int counter = 0; private int id; protected byte buffer[] = new byte[BUFFER_SIZE]; protected int offset = 0; private static int totalBytesWritten = 0; private static int totalCompressedBytesWritten = 0; public CompressingOutputStream(OutputStream os) { this.os = os; deflater = new Deflater(); synchronized (CompressingOutputStream.class) { id = counter++; } } public void write(int b) throws IOException { ensureBufferNotFull(); buffer[offset++] = (byte) b; ensureBufferNotFull(); } public void flush() throws IOException { try { if (offset == 0) { ; } else { deflater.reset(); deflater.setInput(buffer, 0, offset); deflater.finish(); byte [] compressedBuffer = new byte[offset + COMPRESSION_PAD]; deflater.deflate(compressedBuffer); if (!deflater.needsInput()) { throw new IOException("Insufficient pad for compression"); } int compressedSize = (int) deflater.getTotalOut(); PacketHeader header = new PacketHeader((int) deflater.getTotalIn(), (int) compressedSize); logPacket(compressedBuffer, compressedSize); header.write(os); os.write(compressedBuffer, 0, compressedSize); } } finally { offset = 0; } os.flush(); } private void ensureBufferNotFull() throws IOException { if (offset >= BUFFER_SIZE) { flush(); } } protected void logPacket(byte [] compressedBuffer, int compressedSize) { logCompressionRatios(compressedSize); try { StringBuffer sb = new StringBuffer(); sb.append("Uncompressed buffer of size "); sb.append(offset); if (compressedSize > offset) { sb.append(" (compression increased size)"); } sb.append(": "); for (int i = 0; i < offset; i++) { sb.append(buffer[i]); sb.append(" "); } sb = new StringBuffer(); sb.append("Compressed buffer of size "); sb.append(compressedSize); sb.append(": "); for (int i = 0; i < compressedSize; i++) { sb.append(compressedBuffer[i]); sb.append(" "); } } catch (Throwable t) { } } private void logCompressionRatios(int compressedSize) { synchronized (CompressingOutputStream.class) { int previousMB = totalBytesWritten / (KB * KB); totalBytesWritten += offset; totalCompressedBytesWritten += compressedSize; if (previousMB < (totalBytesWritten / (KB * KB))) { } } } }