推荐：FastDFS－开源分布式文件系统

[happyfish100]

FastDFS是一个开源的分布式文件系统，纯C语言编写。她对文件进行管理，功能包括：文件存储、文件同步、文件访问（文件上传、文件下载）等，解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务，如相册网站、视频网站等等。

详情请参阅：FastDFS简介
FastDFS官方网站：FastDFS官方网站

[KKND2006]

mark

[joan119]

mark

[filec75]

不懂，帮顶

[filec75]

不懂，up

[happyfish100]

FastDFS V1.13 Stable Version Released！

Version 1.13 2008-11-29
* re-calculate group 's free space when one of it's storage servers'
free space increase
* add parameters: sync_interval, sync_start_time and sync_end_time to
storage.conf
* performance enhancement: log to buffer, flush to disk every interval seconds
* standard fds closed by daemon_init: 0(stdin), 1(stdout) and 2(stderr)
* fix bug: pthread_kill sometimes cause core dump when program terminated
* fix bug: sync.c open next binlog cause loop call

[happyfish100]

FastDFS stable版本V1.15 发布

版本1.14开始相同的文件内容只保存一份，其余的都采用链接(link)方式指向该文件。文件内容查重采用hash算法，文件签名包括5部分：文件字节数和4个hash code（共24字节）。文件查重使用了FastDHT（分布式hash系统）。change log如下：

Version 1.15 2009-01-28
* use FastDHT v1.04 client code
* use FastDHT client thread safely

Version 1.14 2009-01-18
* storage/storage_sync.c:
old: if (reader.sync_row_count % 1000 == 0)
new: if (reader.scan_row_count % 2000 == 0)
* little adjustment for common files can be used by FastDHT
* sched_thread.h /.c add global variable g_schedule_flag to quit normally
* shared_func.h / .c add function get_time_item_from_conf
* sched_thread.h /.c support time_base of task
* hash.h / .c add function CRC32, add hash function to support stream hash
* add FastDHT client files in storage/fdht_client/
* create symbol link when the file content is duplicate,
add item "check_file_duplicate" to conf file storage.conf
* use FastDHT v1.02 client code
* auto delete invalid entry in FastDHT when the source file does not exist

[mrshelly]

赞......

不知道在 文件key 存储上有没有分布. 没有仔细看源码.

只看过 MogileFS 的源码. MogileFS 在 文件key 存储上,还是有蛮多不好的地方.

[happyfish100]

如果不使用文件查重（相同内容的多个文件只保存一份），FastDFS本身不需要存储文件索引信息的。
FastDHT（分布式Hash系统）也是我做的一个开源项目，详情请参阅：http://fastdht.csource.org/
google code: http://code.google.com/p/fastdht/

[Kelvin]

非常不错，伯乐族 www.ibole.cn 收录了。

伯乐族 www.ibole.cn -- 程序员的世界很精彩

[wanghi]

边学边mark

[ten789]

好东西啊

[happyfish100]

FastDFS stable版本V1.16 发布

Version 1.16 2009-02-14
* client can specify group name when upload file
* tracker_service.c: cmd dispatch changed to "switch ... case"
not "if ... else if"
* storage_service.c: call fdfs_quit before tracker_disconnect_server

[happyfish100]

FastDFS stable版本V1.20 发布

Version 1.20 2009-08-30
* base64 use context, functions changed
* common/ini_file_reader.c: fix memory leak
* tracker server support HTTP protocol, one thread mode
* storage server support HTTP protocol, one thread mode
* fix bug: storage server rebuild, auto sync data correctly
* fix bug: sync data fail (correct storage server status)
* when storage server idle time exceeds check_active_interval seconds,
set it's status to offline

Version 1.19 2009-07-23
* use poll instead of select in sockopt.c
* hash.c use chain impl by self
* use FastDHT 1.09 client code
* ini reader support HTTP protocol, conf file can be an url
* correct test dir compile error
* use non-block socket to increase network IO performance
* add cmd TRACKER_PROTO_CMD_SERVICE_QUERY_FETCH_ALL: query all storage servers
from which the file can be dowloaded
* while (1) ... break; changed to do ... while (0);

Version 1.18 2009-05-24
* restart.sh only kill the programs match the program name and all parameters
* correct get local ip addresses
* common files do not use global vars like g_network_timeout and g_base_path
* download file support offset and download bytes
* hash function change type from unsigned int to signed int
* file size in file name support 64 bits, old bytes is 4, new bytes is 8

Version 1.17 2009-03-19
* add test programs at sub directory test/
* common/shared_func.c: rindex change to strrchr, add #include <netinet/in.h>
* support SunOS (Solaris), compile passed on SunOS 5.10
* support AIX, compile passed on AIX 5.3
* sys call statfs change to statvfs
* use scheduling thread to sync binlog buff / cache to disk, add parameter
"sync_binlog_buff_interval" to conf file storage.conf
* use FastDHT v1.07 client code

[yhp]

TO happyfish100:
能不能把FastDFS和Hadoop做个比较啊？hadoop有个很大的缺陷是，Namenode只能配置一台，存在单点失败的问题。

[thinke365]

LZ仔细研究过MogileFS的代码吧，呵呵

[saturn6120]

mark

[tyzqqq]

ftp比这个功能如何

[Ivy就是长青草]

这个很不错！我最近正在看。。。论坛回复挺及时的，谢谢！（我怎么这么像托啊。。。）

FastDFS官方网站：FastDFS官方网站

[ynhhjpljx]

学习学习

[ynhhjpljx]

支持开源~~