Why use logFileOffset as endOffset of logfile?

[Edward (Yang)]

In raftwal/wal.go:

func (l *wal) AddEntries(entries []raftpb.Entry) error {
    ...
     z.ZeroOut(l.current.Data, entrySize*eidx, logFileOffset)
     ...
}

if I want to clear all the entry after the eidx, why use logFileOffset as end of file?
why not use

    z.ZeroOut(l.current.Data, entrySize*eidx, len(l.current.Data))

In raftwal/log.go:

// logFileOffset is offset in the log file where data is stored.
logFileOffset = 1 << 20 // 1MB

logFileOffset just define 1MB, why can use it as indicator as end of file?

[chewxy]

So the part where ZeroOut is called with that line in wal.go only happens if it is there is already an entry in the previous file. So we go to the previous file, and find the entry, and zero out from that point onwards to the end of the file. All the subsequent files get deleted. You literally want to zero out eidx to the end of the file

[Edward (Yang)]

I know your meaning, but my question is why use LogFileOffset as the end of zeroout.
I understand logFileOffset is the begin position of first entry in the log file.
If I want clear the data range like [3M, file_end),
I will write z.ZeroOut(l.current.Data, 3M, len(l.current.Data)),
why z.ZeroOut(l.current.Data, 3M, logFileOffset) work???

[mrjn (Manish R Jain)]

we use fixed entries to represent the raft entries in the first 1MB of the file. They contain offset to the data beyond logfileOffset. We don’t need to zero those out. We just zero out the entries, and the data offsets would get overwritten by new entries.

[Edward (Yang)]

Maybe I see you. you mean before 1MB, logfile only save fixed numbers entries’ offset. After 1MB, it save real all entries’ data.
So before 1MB, it likes metadata for all entries. So zeroout before 1MB metadata is ok.

[Edward (Yang)]

closed