src/fileops.c


Log

Author Commit Date CI Message
Vicent Marti 72a3fe42 2011-03-18T19:38:49 I broke your bindings Hey. Apologies in advance -- I broke your bindings. This is a major commit that includes a long-overdue redesign of the whole object-database structure. This is expected to be the last major external API redesign of the library until the first non-alpha release. Please get your bindings up to date with these changes. They will be included in the next minor release. Sorry again! Major features include: - Real caching and refcounting on parsed objects - Real caching and refcounting on objects read from the ODB - Streaming writes & reads from the ODB - Single-method writes for all object types - The external API is now partially thread-safe The speed increases are significant in all aspects, specially when reading an object several times from the ODB (revwalking) and when writing big objects to the ODB. Here's a full changelog for the external API: blob.h ------ - Remove `git_blob_new` - Remove `git_blob_set_rawcontent` - Remove `git_blob_set_rawcontent_fromfile` - Rename `git_blob_writefile` -> `git_blob_create_fromfile` - Change `git_blob_create_fromfile`: The `path` argument is now relative to the repository's working dir - Add `git_blob_create_frombuffer` commit.h -------- - Remove `git_commit_new` - Remove `git_commit_add_parent` - Remove `git_commit_set_message` - Remove `git_commit_set_committer` - Remove `git_commit_set_author` - Remove `git_commit_set_tree` - Add `git_commit_create` - Add `git_commit_create_v` - Add `git_commit_create_o` - Add `git_commit_create_ov` tag.h ----- - Remove `git_tag_new` - Remove `git_tag_set_target` - Remove `git_tag_set_name` - Remove `git_tag_set_tagger` - Remove `git_tag_set_message` - Add `git_tag_create` - Add `git_tag_create_o` tree.h ------ - Change `git_tree_entry_2object`: New signature is `(git_object **object_out, git_repository *repo, git_tree_entry *entry)` - Remove `git_tree_new` - Remove `git_tree_add_entry` - Remove `git_tree_remove_entry_byindex` - Remove `git_tree_remove_entry_byname` - Remove `git_tree_clearentries` - Remove `git_tree_entry_set_id` - Remove `git_tree_entry_set_name` - Remove `git_tree_entry_set_attributes` object.h ------------ - Remove `git_object_new - Remove `git_object_write` - Change `git_object_close`: This method is now *mandatory*. Not closing an object causes a memory leak. odb.h ----- - Remove type `git_rawobj` - Remove `git_rawobj_close` - Rename `git_rawobj_hash` -> `git_odb_hash` - Change `git_odb_hash`: New signature is `(git_oid *id, const void *data, size_t len, git_otype type)` - Add type `git_odb_object` - Add `git_odb_object_close` - Change `git_odb_read`: New signature is `(git_odb_object **out, git_odb *db, const git_oid *id)` - Change `git_odb_read_header`: New signature is `(size_t *len_p, git_otype *type_p, git_odb *db, const git_oid *id)` - Remove `git_odb_write` - Add `git_odb_open_wstream` - Add `git_odb_open_rstream` odb_backend.h ------------- - Change type `git_odb_backend`: New internal signatures are as follows int (* read)(void **, size_t *, git_otype *, struct git_odb_backend *, const git_oid *) int (* read_header)(size_t *, git_otype *, struct git_odb_backend *, const git_oid *) int (* writestream)(struct git_odb_stream **, struct git_odb_backend *, size_t, git_otype) int (* readstream)( struct git_odb_stream **, struct git_odb_backend *, const git_oid *) - Add type `git_odb_stream` - Add enum `git_odb_streammode` Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 55ffebe3 2011-03-05T14:34:32 Fix creation of deeply-rooted references Use a new `gitfo_creat_force` that will create the full path to a file before creating it. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 19a30a3f 2011-03-03T19:53:17 Add new move function, `gitfo_mv_force` Forces a move by creating the folder for the destination file, if it doesn't exist. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti ccef1c9d 2011-02-27T22:09:36 Move the path comparison method to fileops.c Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 7360122b 2011-02-24T23:53:40 Fix file renaming in MinGW We now use MoveFileEx, which is not assured to be atomic but works for always (both if the destination exists, or if it doesn't) and is available in MinGW. Since this is a Win32 API call, complaint about lost or overwritten files should be forwarded at Steve Ballmer. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 3eb47c9f 2011-02-24T21:43:08 Fix renaming of files in Win32 The `rename` call doesn't quite work on Win32: expects the destination file to not exist. We're using a native Win32 call in those cases -- that should do the trick. Signed-off-by: Vicent Marti <tanoku@gmail.com>
nulltoken f2c24713 2011-01-26T20:29:06 Made path prettifying functions return GIT_EINVALIDPATH instead of GIT_ERROR.
nulltoken 2e6fd09c 2011-01-25T21:52:24 Fixed naming convention related issue.
nulltoken 618818dc 2011-01-23T16:15:11 Added git_prettify_file_path().
nulltoken 4581c22a 2011-01-22T14:38:47 Optimized git_prettify_dir_path() parsing.
nulltoken ae7ffea9 2011-01-22T14:04:32 Fixed a parsing issue in git_prettify_dir_path().
nulltoken e16c2f6a 2011-01-20T19:51:34 Small enhancements to git_prettify_dir_path(). - Secured buffer ahead reading. - Guard against potential multiple dot path traversal (cf http://cwe.mitre.org/data/definitions/33.html)
Vicent Marti b5e567b9 2011-01-19T17:15:02 Merge branch 'dir-path-prettifying' of https://github.com/nulltoken/libgit2
nulltoken 170d3f2f 2011-01-11T20:12:53 Added git_prettify_dir_path(). Clean up a provided absolute or relative directory path. This prettification relies on basic operations such as coalescing multiple forward slashes into a single slash, removing '.' and './' current directory segments, and removing parent directory whenever '..' is encountered. If not empty, the returned path ends with a forward slash. For instance, this will turn "d1/s1///s2/..//../s3" into "d1/s3/". This only performs a string based analysis of the path. No checks are done to make sure the path actually makes sense from the file system perspective.
Alex Budovski f0bde7fa 2011-01-11T16:07:45 Revised platform types to use 'best supported' size. This will allow graceful migration to 64 bit file sizes and timestamps should git's binary interface be extended to allow this.
Vicent Marti d5f25204 2011-01-03T21:37:14 Merge branch 'repo-init' of https://github.com/nulltoken/libgit2 into nulltoken-repo-init
nulltoken 951d06e4 2010-12-26T17:00:35 Fixed placement of pointer argument.
nulltoken 98e3b298 2010-12-23T09:22:15 Merge branch 'master' into repo-init
Vicent Marti 9f54fe48 2010-12-23T00:15:09 Remove git_errno It was not being used by any methods (only by malloc and calloc), and since it needs to be TLS, it cannot be exported on DLLs on Windows. Burn it with fire. The API always returns error codes! Signed-off-by: Vicent Marti <tanoku@gmail.com>
nulltoken 2c08c3f0 2010-12-21T06:52:07 Made is_windows_rooted_path() able to cope with awkward but valid relative paths such as "C:..\File.txt". Path "C:..\File.txt" refers to a file called File.txt located in the parent directory of the current directory on drive C:.
nulltoken 23a1edbd 2010-12-21T05:43:47 Wrapped the detection of a Windows rooted path within a conditional compilation directive.
nulltoken 2e29957a 2010-12-20T10:56:32 Made gitfo_mkdir_recurs() skip creation of the root of the path if it looks like a Windows drive.
nulltoken 412de9a6 2010-12-20T10:43:23 Made gitfo_mkdir_recurs() gracefully recover when a given directory already exists.
Vicent Marti 40c44d2f 2010-12-19T22:50:20 Fix issues in repository initialization Implemented recursive directory creation Fix style issues Signed-off-by: Vicent Marti <tanoku@gmail.com>
nulltoken d13e7e02 2010-12-17T15:58:30 Added a comment reminding that the gitfo_mkdir_recurs() is far from being fully implemented.
nulltoken f0b2bfe5 2010-12-17T14:24:47 Made gitfo_mkdir_recurs() more tolerant. Doesn't err any more when the directory already exists.
nulltoken 1a5204a7 2010-12-17T10:08:01 Added gitfo_mkdir_recurs() placeholder.
Vicent Marti 1f080e2d 2010-12-13T03:43:56 Fix initialization & freeing of inexistent repos Signed-off-by: Vicent Marti <tanoku@gmail.com>
nulltoken 6f02c3ba 2010-12-05T20:18:56 Small source code readability improvements. Replaced magic number "0" with GIT_SUCCESS constant wherever it made sense.
Vicent Marti 43e380a8 2010-11-23T15:18:53 Fix gitfo_isdir() in Win32 systems Because adhering to the POSIX standards is overrated. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Vicent Marti 6fd195d7 2010-11-02T18:42:42 Change git_repository initialization to use a path The constructor to git_repository is now called 'git_repository_open(path)' and takes a path to a git repository instead of an existing ODB object. Unit tests have been updated accordingly and the two test repositories have been merged into one. Signed-off-by: Vicent Marti <tanoku@gmail.com>
Ramsay Jones 3cc60635 2010-03-05T19:06:28 Add some more (macro) file operation wrappers Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones e8a95256 2010-01-04T18:57:13 msvc: Fix some -W4 warnings Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 803a6b4d 2010-01-03T19:12:05 Save the git_file in the gitfo_cache when enabling caching Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 90d4d2f0 2010-01-11T19:27:50 win32: Use an 64-bit file offset type Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Ramsay Jones 4c9a3973 2009-06-04T17:13:49 Improve the portability of the gitfo_cache code (to 64-bit systems) This supresses some "conversion from 'size_t' to 'unsigned int', possible loss of data" warning messages from the MS Visual C/C++ compiler with -Wp64. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones ca481fc4 2009-06-04T16:46:54 Add the git_move_file() function Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones 4d503f88 2009-06-04T16:48:53 gitfo_exists: replace call to stat() with access() Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Andreas Ericsson <ae@op5.se>
Ramsay Jones 79ca2edc 2009-03-20T19:51:48 win32: Add routines to abstract memory-mapped file functions In particular, the git__mmap() and git__munmap() routines provide the interface to platform specific memory-mapped file facilities. We provide implementations for unix and win32, which can be found in their own sub-directories. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 7a6cf815 2009-03-16T17:08:45 win32: Open or create files in binary mode On windows, unless we use the O_BINARY flag in the open() call, the file I/O routines will perform line ending conversion (\r\n => \n on input, \n => \r\n on output). In addition to the performance penalty, most files in the object database are binary and will, therefore, become corrupted by this conversion. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 840fb8b7 2009-02-18T18:53:48 win32: fixup some headers to improve win32 compilation Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones a9984a4e 2009-02-18T18:52:13 Fix some (digital-mars) compiler warnings In particular, conditional expressions which contain an assignment statement, where the expression type is not explicitly made to be boolean, elicits the following message: warning 2: possible unintended assignment Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones fc3c3a20 2009-02-18T18:50:36 Avoid using a gcc compiler language extension In particular, using pointer arithmetic on void pointers, despite being quite useful, is not legal in standard C. Avoiding non-standard C constructs will help in porting the library to other compilers/platforms. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 20e7f426 2009-01-03T00:36:10 Add a simple mmap wrapper for cross-platform mmap usage Win32 has a variant of mmap that is harder to use than POSIX, but to run natively and efficiently on Win32 we need some form of it. gitfo_map_ro() provides a basic mmap function for use in locations where we need read-only random data access to large ranges of a file, such as a pack-*.idx. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 7350e633 2008-12-31T16:07:38 Define gitfo_exists to determine file presence When scanning the pack directory we need to see if the path name is present for ".idx" when we discover a ".pack" file. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 5690f02e 2008-12-31T15:35:36 Rewrite git_foreach_dirent into gitfo_dirent Our fileops API is currently private. We aren't planning on supplying a cross-platform file API to applications that link to us. If we did, we'd probably whole-sale publish fileops, not just the dirent code. By moving it to be private we can also change the call signature to permit the buffer to be passed down through the call chain. This is very helpful when we are doing a recursive scan as we can reuse just one buffer in all stack frames, reducing the impact the recursion has on the stack frames in the data cache. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 7dd8a9f7 2008-12-30T23:26:38 Set GIT_EOSERR when the OS errno should be consulted This error code indicates the OS error code has a better value describing the last error, as it is likely a network or local file IO problem identified by a C library function call. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce 64a47c01 2008-12-30T23:21:36 Wrap malloc and friends and report out of memory as GIT_ENOMEM We now forbid direct use of malloc, strdup or calloc within the library and instead use wrapper functions git__malloc, etc. to invoke the underlying library malloc and set git_errno to a no memory error code if the allocation fails. In the future once we have pack objects in memory we are likely to enhance these routines with garbage collection logic to purge cached pack data when allocations fail. Because the size of the function will grow somewhat large, we don't want to mark them for inline as gcc tends to aggressively inline, creating larger than expected executables. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Julio Espinoza-Sokal 213e720c 2008-12-20T20:47:41 Change usages of static inline to GIT_INLINE Signed-off-by: Julio Espinoza-Sokal <julioes@gmail.com> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 42fd40db 2008-12-27T18:56:16 Fix a bug in gitfo_read_file() In particular, when asked to read an empty file, this function calls malloc() with a zero size allocation request. Standard C says that the behaviour of malloc() in this case is implementation defined. [C99, 7.20.3 says "... If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object."] Finesse the issue by over-allocating by one byte. Setting the extra byte to '\0' may also provide a useful sentinel for text files. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 75d58430 2008-12-18T22:56:14 Add a file reading routine along with an io buffer type In particular, the gitfo_read_file() routine can be used to slurp the complete file contents into an gitfo_buf structure. The buffer content will be allocated by malloc() and may be released by the gitfo_free_buf() routine. The io buffer type can be initialised on the stack with the GITFO_BUF_INIT macro. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Ramsay Jones 5ee2fe77 2008-12-03T23:53:55 Add a GIT_PATH_MAX constant The PATH_MAX symbol is often, but not always, defined in the <limits.h> header. In particular, on cygwin you need to include this header to avoid a compilation error. However, some systems define PATH_MAX to be something as small as 256, which POSIX is happy to allow, while others allow much larger values. In general it can vary from one filesystem to another. In order to avoid the vagaries of different systems, define our own symbol. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Andreas Ericsson ea790f33 2008-11-29T15:34:20 Add a dirent walker to the fileops API Since at least MS have something like GetFirstDirEnt() and GetNextDirEnt() (presumably with superior performance), we can let MS hackers add support for a dirent walker using that API instead, while we stick with the posix-style readdir() calls. Signed-off-by: Andreas Ericsson <ae@op5.se> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Andreas Ericsson 4188d28f 2008-11-29T15:28:12 Add an io caching layer to the gitfo api The idea is taken from Junio's work in read-cache.c, where it's used for writing out the index without tap-dancing on the poor harddrive. Since it's almost certainly useful for cached writing of packfiles too, we turn it into a generic API, making it perfectly simple to reuse it later. gitfo_write_cached() has the same contract as gitfo_write(), it returns GIT_SUCCESS if all bytes are successfully written (or were at least buffered for later writing), and <0 if an error occurs during buffer writing. Signed-off-by: Andreas Ericsson <ae@op5.se> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Andreas Ericsson ec250c6e 2008-11-23T22:37:55 Remove config.h and make fileops an internal API Since it doesn't make sense to make the disk access stuff portable *AND* public (that's a job for each application imo), we can take a shortcut and just support unixy stuff for now and get away with coding most of it as macros. Since we go with an internal API for starters and only provide higher-level API's to the libgit users, we'll be ok with this approach. Signed-off-by: Andreas Ericsson <ae@op5.se> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>