kmx git

Commit	Date	Message
afeecf4f	2011-07-09T02:10:46	odb: Direct writes are back DIRECT WRITES ARE BACK AND FASTER THAN EVER. The streaming writer to the ODB was an overkill for the smaller objects like Commit and Tags; most of the streaming logic was taking too long. This commit makes Commits, Tags and Trees to be built-up in memory, and then written to disk in 2 pushes (header + data), instead of streaming everything. This is always faster, even for big files (since the git_filebuf class still does streaming writes when the memory cache overflows). This is also a gazillion lines of code smaller, because we don't have to precompute the final size of the object before starting the stream (this was kind of defeating the point of streaming, anyway). Blobs are still written with full streaming instead of loading them in memory, since this is still the fastest way. A new `git_buf` class has been added. It's missing some features, but it'll get there.
2fc78e70	2011-07-08T23:01:37	posix: Portable `vsnprintf` Our good, lovely folks at Microsoft decided that there was no good reason to make `vsnprintf` compilant with the C standard, so that function in Windows returns -1 on overflow, instead of returning the actual byte count needed to write the full string. We now handle this situation more gracefully with the POSIX compatibility layer, by returning the needed byte size using an auxiliary method instead of blindly resizing the target buffer until it fits. This means we can now support `printf`s of any size by allocating a temporary buffer. That's good.
f79026b4	2011-07-04T11:43:34	fileops: Cleanup Cleaned up the structure of the whole OS-abstraction layer. fileops.c now contains a set of utility methods for file management used by the library. These are abstractions on top of the original POSIX calls. There's a new file called `posix.c` that contains emulations/reimplementations of all the POSIX calls the library uses. These are prefixed with `p_`. There's a specific posix file for each platform (win32 and unix). All the path-related methods have been moved from `utils.c` to `path.c` and have their own prefix.
932d1baf	2011-06-30T19:52:34	cleanup: remove trailing spaces Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
fe5babac	2011-06-30T00:16:23	filebuf: fix endless loop on writing buf > WRITE_BUFFER_SIZE Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
bb9272dd	2011-06-07T17:03:07	filebuf cleanup: only unlink lockfile if we've opened it Add a check for the file descriptor in git_filebuf_cleanup. Without it, an existing lockfile would be deleted if we tried to acquire it (but failed, as the lockfile already existed). Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
f9213015	2011-06-06T01:54:59	filebuf: Fix printf buffer overflows The filebuf was not being properly written after a flush. This should cut it now. Fixes #228
1549cba9	2011-06-03T21:18:24	Filebuf: Fixed a TODO in filebuf (real lock in lock_file) Added gitfo_creat_locked and gitfo_creat_locked_force
f6328611	2011-05-23T21:00:19	filebuf: Reword errors
374db5f9	2011-05-17T17:43:00	filebuf.c: Move to new error handling mechanism
a6359408	2011-04-10T12:23:55	Use Z_BEST_SPEED for filebuf deflating This is what Git uses by default for all deflating.
f6f72d7e	2011-03-23T18:44:53	Improve the ODB writing backend Temporary files when doing streaming writes are now stored inside the Objects folder, to prevent issues when moving files between disks/partitions. Add support for block writes to the ODB again (for those backends that cannot implement streaming).
72a3fe42	2011-03-18T19:38:49	I broke your bindings Hey. Apologies in advance -- I broke your bindings. This is a major commit that includes a long-overdue redesign of the whole object-database structure. This is expected to be the last major external API redesign of the library until the first non-alpha release. Please get your bindings up to date with these changes. They will be included in the next minor release. Sorry again! Major features include: - Real caching and refcounting on parsed objects - Real caching and refcounting on objects read from the ODB - Streaming writes & reads from the ODB - Single-method writes for all object types - The external API is now partially thread-safe The speed increases are significant in all aspects, specially when reading an object several times from the ODB (revwalking) and when writing big objects to the ODB. Here's a full changelog for the external API: blob.h ------ - Remove `git_blob_new` - Remove `git_blob_set_rawcontent` - Remove `git_blob_set_rawcontent_fromfile` - Rename `git_blob_writefile` -> `git_blob_create_fromfile` - Change `git_blob_create_fromfile`: The `path` argument is now relative to the repository's working dir - Add `git_blob_create_frombuffer` commit.h -------- - Remove `git_commit_new` - Remove `git_commit_add_parent` - Remove `git_commit_set_message` - Remove `git_commit_set_committer` - Remove `git_commit_set_author` - Remove `git_commit_set_tree` - Add `git_commit_create` - Add `git_commit_create_v` - Add `git_commit_create_o` - Add `git_commit_create_ov` tag.h ----- - Remove `git_tag_new` - Remove `git_tag_set_target` - Remove `git_tag_set_name` - Remove `git_tag_set_tagger` - Remove `git_tag_set_message` - Add `git_tag_create` - Add `git_tag_create_o` tree.h ------ - Change `git_tree_entry_2object`: New signature is `(git_object *object_out, git_repository repo, git_tree_entry entry)` - Remove `git_tree_new` - Remove `git_tree_add_entry` - Remove `git_tree_remove_entry_byindex` - Remove `git_tree_remove_entry_byname` - Remove `git_tree_clearentries` - Remove `git_tree_entry_set_id` - Remove `git_tree_entry_set_name` - Remove `git_tree_entry_set_attributes` object.h ------------ - Remove `git_object_new - Remove `git_object_write` - Change `git_object_close`: This method is now mandatory. Not closing an object causes a memory leak. odb.h ----- - Remove type `git_rawobj` - Remove `git_rawobj_close` - Rename `git_rawobj_hash` -> `git_odb_hash` - Change `git_odb_hash`: New signature is `(git_oid id, const void data, size_t len, git_otype type)` - Add type `git_odb_object` - Add `git_odb_object_close` - Change `git_odb_read`: New signature is `(git_odb_object out, git_odb db, const git_oid id)` - Change `git_odb_read_header`: New signature is `(size_t len_p, git_otype type_p, git_odb db, const git_oid id)` - Remove `git_odb_write` - Add `git_odb_open_wstream` - Add `git_odb_open_rstream` odb_backend.h ------------- - Change type `git_odb_backend`: New internal signatures are as follows int ( read)(void *, size_t , git_otype , struct git_odb_backend , const git_oid ) int ( read_header)(size_t , git_otype , struct git_odb_backend , const git_oid ) int (* writestream)(struct git_odb_stream *, struct git_odb_backend , size_t, git_otype) int (* readstream)( struct git_odb_stream *, struct git_odb_backend , const git_oid *) - Add type `git_odb_stream` - Add enum `git_odb_streammode` Signed-off-by: Vicent Marti <tanoku@gmail.com>
55ffebe3	2011-03-05T14:34:32	Fix creation of deeply-rooted references Use a new `gitfo_creat_force` that will create the full path to a file before creating it. Signed-off-by: Vicent Marti <tanoku@gmail.com>
19a30a3f	2011-03-03T19:53:17	Add new move function, `gitfo_mv_force` Forces a move by creating the folder for the destination file, if it doesn't exist. Signed-off-by: Vicent Marti <tanoku@gmail.com>
86194b24	2011-02-18T21:57:53	Split packed from unpacked references These two reference types are now stored separately to eventually allow the removal/renaming of loose references and rewriting of the refs packfile. Signed-off-by: Vicent Marti <tanoku@gmail.com>
c3be1468	2011-02-24T19:31:12	Fix double-freeing file descriptors Was crashing the Windows build. Signed-off-by: Vicent Marti <tanoku@gmail.com>
5591ea15	2011-02-22T14:58:54	Add printf method to the File Buffer Signed-off-by: Vicent Marti <tanoku@gmail.com>
817c2820	2011-02-21T17:05:16	Rewrite all file IO for more performance The new `git_filebuf` structure provides atomic high-performance writes to disk by using a write cache, and optionally a double-buffered scheme through a worker thread (not enabled yet). Writes can be done 3-layered, like in git.git (user code -> write cache -> disk), or 2-layered, by writing directly on the cache. This makes index writing considerably faster. The `git_filebuf` structure contains all the old functionality of `git_filelock` for atomic file writes and reads. The `git_filelock` structure has been removed. Additionally, the `git_filebuf` API allows to automatically hash (SHA1) all the data as it is written to disk (hashing is done smartly on big chunks to improve performance). Signed-off-by: Vicent Marti <tanoku@gmail.com>

afeecf4f

2011-07-09T02:10:46

odb: Direct writes are back DIRECT WRITES ARE BACK AND FASTER THAN EVER. The streaming writer to the ODB was an overkill for the smaller objects like Commit and Tags; most of the streaming logic was taking too long. This commit makes Commits, Tags and Trees to be built-up in memory, and then written to disk in 2 pushes (header + data), instead of streaming everything. This is *always* faster, even for big files (since the git_filebuf class still does streaming writes when the memory cache overflows). This is also a gazillion lines of code smaller, because we don't have to precompute the final size of the object before starting the stream (this was kind of defeating the point of streaming, anyway). Blobs are still written with full streaming instead of loading them in memory, since this is still the fastest way. A new `git_buf` class has been added. It's missing some features, but it'll get there.

2fc78e70

2011-07-08T23:01:37

posix: Portable `vsnprintf` Our good, lovely folks at Microsoft decided that there was no good reason to make `vsnprintf` compilant with the C standard, so that function in Windows returns -1 on overflow, instead of returning the actual byte count needed to write the full string. We now handle this situation more gracefully with the POSIX compatibility layer, by returning the needed byte size using an auxiliary method instead of blindly resizing the target buffer until it fits. This means we can now support `printf`s of any size by allocating a temporary buffer. That's good.

f79026b4

2011-07-04T11:43:34

fileops: Cleanup Cleaned up the structure of the whole OS-abstraction layer. fileops.c now contains a set of utility methods for file management used by the library. These are abstractions on top of the original POSIX calls. There's a new file called `posix.c` that contains emulations/reimplementations of all the POSIX calls the library uses. These are prefixed with `p_`. There's a specific posix file for each platform (win32 and unix). All the path-related methods have been moved from `utils.c` to `path.c` and have their own prefix.

932d1baf

2011-06-30T19:52:34

cleanup: remove trailing spaces Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>

fe5babac

2011-06-30T00:16:23

filebuf: fix endless loop on writing buf > WRITE_BUFFER_SIZE Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>

bb9272dd

2011-06-07T17:03:07

filebuf cleanup: only unlink lockfile if we've opened it Add a check for the file descriptor in git_filebuf_cleanup. Without it, an existing lockfile would be deleted if we tried to acquire it (but failed, as the lockfile already existed). Signed-off-by: Carlos Martín Nieto <cmn@elego.de>

f9213015

2011-06-06T01:54:59

filebuf: Fix printf buffer overflows The filebuf was not being properly written after a flush. This should cut it now. Fixes #228

1549cba9

2011-06-03T21:18:24

Filebuf: Fixed a TODO in filebuf (real lock in lock_file) Added gitfo_creat_locked and gitfo_creat_locked_force

f6328611

2011-05-23T21:00:19

filebuf: Reword errors

374db5f9

2011-05-17T17:43:00

filebuf.c: Move to new error handling mechanism

a6359408

2011-04-10T12:23:55

Use Z_BEST_SPEED for filebuf deflating This is what Git uses by default for all deflating.

f6f72d7e

2011-03-23T18:44:53

Improve the ODB writing backend Temporary files when doing streaming writes are now stored inside the Objects folder, to prevent issues when moving files between disks/partitions. Add support for block writes to the ODB again (for those backends that cannot implement streaming).

72a3fe42

2011-03-18T19:38:49

I broke your bindings Hey. Apologies in advance -- I broke your bindings. This is a major commit that includes a long-overdue redesign of the whole object-database structure. This is expected to be the last major external API redesign of the library until the first non-alpha release. Please get your bindings up to date with these changes. They will be included in the next minor release. Sorry again! Major features include: - Real caching and refcounting on parsed objects - Real caching and refcounting on objects read from the ODB - Streaming writes & reads from the ODB - Single-method writes for all object types - The external API is now partially thread-safe The speed increases are significant in all aspects, specially when reading an object several times from the ODB (revwalking) and when writing big objects to the ODB. Here's a full changelog for the external API: blob.h ------ - Remove `git_blob_new` - Remove `git_blob_set_rawcontent` - Remove `git_blob_set_rawcontent_fromfile` - Rename `git_blob_writefile` -> `git_blob_create_fromfile` - Change `git_blob_create_fromfile`: The `path` argument is now relative to the repository's working dir - Add `git_blob_create_frombuffer` commit.h -------- - Remove `git_commit_new` - Remove `git_commit_add_parent` - Remove `git_commit_set_message` - Remove `git_commit_set_committer` - Remove `git_commit_set_author` - Remove `git_commit_set_tree` - Add `git_commit_create` - Add `git_commit_create_v` - Add `git_commit_create_o` - Add `git_commit_create_ov` tag.h ----- - Remove `git_tag_new` - Remove `git_tag_set_target` - Remove `git_tag_set_name` - Remove `git_tag_set_tagger` - Remove `git_tag_set_message` - Add `git_tag_create` - Add `git_tag_create_o` tree.h ------ - Change `git_tree_entry_2object`: New signature is `(git_object **object_out, git_repository *repo, git_tree_entry *entry)` - Remove `git_tree_new` - Remove `git_tree_add_entry` - Remove `git_tree_remove_entry_byindex` - Remove `git_tree_remove_entry_byname` - Remove `git_tree_clearentries` - Remove `git_tree_entry_set_id` - Remove `git_tree_entry_set_name` - Remove `git_tree_entry_set_attributes` object.h ------------ - Remove `git_object_new - Remove `git_object_write` - Change `git_object_close`: This method is now *mandatory*. Not closing an object causes a memory leak. odb.h ----- - Remove type `git_rawobj` - Remove `git_rawobj_close` - Rename `git_rawobj_hash` -> `git_odb_hash` - Change `git_odb_hash`: New signature is `(git_oid *id, const void *data, size_t len, git_otype type)` - Add type `git_odb_object` - Add `git_odb_object_close` - Change `git_odb_read`: New signature is `(git_odb_object **out, git_odb *db, const git_oid *id)` - Change `git_odb_read_header`: New signature is `(size_t *len_p, git_otype *type_p, git_odb *db, const git_oid *id)` - Remove `git_odb_write` - Add `git_odb_open_wstream` - Add `git_odb_open_rstream` odb_backend.h ------------- - Change type `git_odb_backend`: New internal signatures are as follows int (* read)(void **, size_t *, git_otype *, struct git_odb_backend *, const git_oid *) int (* read_header)(size_t *, git_otype *, struct git_odb_backend *, const git_oid *) int (* writestream)(struct git_odb_stream **, struct git_odb_backend *, size_t, git_otype) int (* readstream)( struct git_odb_stream **, struct git_odb_backend *, const git_oid *) - Add type `git_odb_stream` - Add enum `git_odb_streammode` Signed-off-by: Vicent Marti <tanoku@gmail.com>

55ffebe3

2011-03-05T14:34:32

Fix creation of deeply-rooted references Use a new `gitfo_creat_force` that will create the full path to a file before creating it. Signed-off-by: Vicent Marti <tanoku@gmail.com>

19a30a3f

2011-03-03T19:53:17

Add new move function, `gitfo_mv_force` Forces a move by creating the folder for the destination file, if it doesn't exist. Signed-off-by: Vicent Marti <tanoku@gmail.com>

86194b24

2011-02-18T21:57:53

Split packed from unpacked references These two reference types are now stored separately to eventually allow the removal/renaming of loose references and rewriting of the refs packfile. Signed-off-by: Vicent Marti <tanoku@gmail.com>

c3be1468

2011-02-24T19:31:12

Fix double-freeing file descriptors Was crashing the Windows build. Signed-off-by: Vicent Marti <tanoku@gmail.com>

5591ea15

2011-02-22T14:58:54

Add printf method to the File Buffer Signed-off-by: Vicent Marti <tanoku@gmail.com>

817c2820

2011-02-21T17:05:16

Rewrite all file IO for more performance The new `git_filebuf` structure provides atomic high-performance writes to disk by using a write cache, and optionally a double-buffered scheme through a worker thread (not enabled yet). Writes can be done 3-layered, like in git.git (user code -> write cache -> disk), or 2-layered, by writing directly on the cache. This makes index writing considerably faster. The `git_filebuf` structure contains all the old functionality of `git_filelock` for atomic file writes and reads. The `git_filelock` structure has been removed. Additionally, the `git_filebuf` API allows to automatically hash (SHA1) all the data as it is written to disk (hashing is done smartly on big chunks to improve performance). Signed-off-by: Vicent Marti <tanoku@gmail.com>

thodg/libgit2/src/filebuf.c

src/filebuf.c

Log