[Closed]fix bug for removing hard link
复现
在FastCFS下执行git clone https://github.com/happyfish100/FastCFS.git
正克隆到 'FastCFS'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 1129 (delta 3), reused 4 (delta 0), pack-reused 1110
接收对象中: 100% (1129/1129), 1.68 MiB | 490.00 KiB/s, done.
处理 delta 中: 100% (701/701), done.
error: wrong index v1 file size in /usr/local/fastcfs-test-standalone/fuse/fuse1/FastCFS/.git/objects/pack/pack-3d4a9f6c578cb3a72a4b2526c0af410fb34ea163.idx
error: wrong index v1 file size in /usr/local/fastcfs-test-standalone/fuse/fuse1/FastCFS/.git/objects/pack/pack-3d4a9f6c578cb3a72a4b2526c0af410fb34ea163.idx
error: wrong index v1 file size in /usr/local/fastcfs-test-standalone/fuse/fuse1/FastCFS/.git/objects/pack/pack-3d4a9f6c578cb3a72a4b2526c0af410fb34ea163.idx
fatal: bad object f3a15bb4acefb676463b92e6d5db756fb14cde93
fatal: 远程没有发送所有必须的对象
Unexpected end of command stream
尝试 git 建仓、添加文件、提交文件
git init foo && cd foo
echo 'test content' > foo.txt
git commit -m 'test'
此时commit失败
error: unable to unpack 7cb102b7d7bfe665f816404583b575cdb883e87b header
error: inflateEnd: stream consistency error (no message)
fatal: 7cb102b7d7bfe665f816404583b575cdb883e87b is not a valid object
分析
结合报错日志wrong index v1 file size, 初步认为是文件元数据出了问题。关闭FastDIR的异步报告将fuse.conf的async_report_enabled设置为false。
# if async report file attributes (size, modify time etc.) to the FastDIR server
# default value is true
async_report_enabled = false
关闭重启FastCFS服务，问题仍然存在。
开始查找git的文档理解git内部原理。阅读 10.2 Git 内部原理 - Git 对象后，开始试探。
echo 'test content' | git hash-object -w --stdin
# d670460b4b4aece5915caf5c68d12f560a9fe3e4
find .git/objects -type f
# .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4
git cat-file -p d670460b4b4aece5915caf5c68d12f560a9fe3e4
# error: unable to unpack d670460b4b4aece5915caf5c68d12f560a9fe3e4 header
# error: inflateEnd: stream consistency error (no message)
# fatal: Not a valid object name d670460b4b4aece5915caf5c68d12f560a9fe3e4
将在本地文件系统执行一样的操作得到.git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4与FastCFS的objetcs对象对比。
file .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4
# FastCFS: .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4: data
# Local FS: .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4: VAX COFF executable not stripped - version 737
hexdump .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4
# FastCFS
# 0000000 0000 0000 0000 0000 0000 0000 0000 0000
# 0000010 0000 0000 0000 0000 0000 0000 0000
# 000001d
# Local FS
# 0000000 0178 ca4b 4fc9 3052 6634 4928 2e2d 4851
# 0000010 cfce 492b 2bcd 02e1 4b00 07df 0009
# 000001d
明显两个object文件不一致。此时陷入沉思。将此问题反馈给@happyfish100。
回顾性分析
用echo 'test content' | git hash-object -w --stdin创建一个数据对象并将它手动存到git数据库，
test content的hash-object为d670460b4b4aece5915caf5c68d12f560a9fe3e4,
所以Git会在.git/object目录下创建d6,d6下创建文件70460b4b4aece5915caf5c68d12f560a9fe3e41。
同时查看FastDIR的binlog
0178<rec dv=815 id=9007199258741264 op=2,cr ts=1616335625 ns=2,fs pt=9007199258741261 nm=2,d6 hc=3277 md=16877 bt=1616335625 at=1616335625 ct=1616335625 mt=1616335625 ui=0 gi=0/rec>
0191<rec dv=816 id=9007199258741265 op=2,cr ts=1616335625 ns=2,fs pt=9007199258741264 nm=14,tmp_obj_IpJax6 hc=3277 md=33060 bt=1616335625 at=1616335625 ct=1616335625 mt=1616335625 ui=0 gi=0/rec>
0085<rec dv=817 id=9007199258741265 op=2,up ts=1616335625 hc=3277 sz=29 se=29 ia=29/rec>
0237<rec dv=818 id=9007199258741266 op=2,cr ts=1616335625 ns=2,fs pt=9007199258741264 nm=38,70460b4b4aece5915caf5c68d12f560a9fe3e4 hc=3277 md=4227583 bt=1616335625 at=1616335625 ct=1616335625 mt=1616335625 ui=0 gi=0 si=9007199258741265/rec>
0116<rec dv=819 id=9007199258741265 op=2,rm ts=1616335625 ns=2,fs pt=9007199258741264 nm=14,tmp_obj_IpJax6 hc=3277/rec>
即动作如下
dv 815: create d6
dv 816: create tmp_obj_IpJax6
dv 817: up
dv 818: create 70460b4b4aece5915caf5c68d12f560a9fe3e4 (由tmp_obj_IpJax6硬链接的生成？)
dv 819: rm tmp_obj_IpJax6
此时猜测git是先创建临时文件tmp_obj_IpJax6并写入内容,
然后创建Hard Link 文件70460b4b4aece5915caf5c68d12f560a9fe3e4，
最后删除 tmp_obj_IpJax6。
验证
echo 'test content' > foo.txt
ln foo.txt bar.txt
rm foo.txt
正常情况下cat bar.txt应该输出内容test content，但在FastCFS下没有。
结论
FastCFS 没有正确处理 Hard Link 导致 git 不是正常使用。当然，FastCFS的作者很快就修复了这个问题。