Commit graph

1035 commits

Author SHA1 Message Date
Hallvard Furuseth
579ad14d7c ITS#8156 Fix MDB_MAXKEYSIZE doc 2015-05-28 20:51:24 +02:00
Hallvard Furuseth
aa16d7bc0a ITS#8157 mdb_txn_renew0(): init after error checks 2015-05-27 22:26:54 +02:00
Hallvard Furuseth
8d1f6ca782 mdb_txn_renew(): Drop pointless diff from mdb.RE.
Makes the code friendlier to "git cherry-pick".
2015-05-27 22:23:50 +02:00
Hallvard Furuseth
02e8de5edb Set/clear mp_pad, md_pad (MDB_DUPFIXED data size).
mdb_xcursor_init1(): md_pad is only used when MDB_DUPFIXED.
mdb_page_split():    Copy mp_pad too. Used by mdb_page_list().
2015-05-20 04:04:38 +02:00
Hallvard Furuseth
77bdbdd9de mtest*.c: Fix MDB_NOOVERWRITE, plug cursor leak 2015-05-20 03:17:47 +02:00
Hallvard Furuseth
9bb915675a ITS#8117 Fix MDB_INTEGERDUP keysize doc + md_dcmp 2015-05-19 20:33:35 +02:00
Howard Chu
329e12e4c5 ITS#8117 better fix
Don't change mc_dbx because we would need to undo the change if
the txn aborts. Make the fix (for get) match existing code for put.
2015-05-03 07:57:22 +01:00
Howard Chu
3d2202f31c Revert "ITS#8117 cleanup prev commit"
This reverts commit ce7ba0ceb6.

Revert "ITS#8117 fix INTEGERDUP compare"

This reverts commit ed13d4aa59.
2015-05-03 07:56:40 +01:00
Howard Chu
ce7ba0ceb6 ITS#8117 cleanup prev commit 2015-04-29 17:28:21 +01:00
Howard Chu
ed13d4aa59 ITS#8117 fix INTEGERDUP compare 2015-04-29 17:13:55 +01:00
Howard Chu
f4cab2994f ITS#8109 fix mdb_cursor_del0 on empty DB 2015-04-23 05:49:56 +01:00
Howard Chu
4c62fde880 ITS#8106 retry writes on EINTR 2015-04-17 18:32:54 +01:00
Howard Chu
ccf4d8b385 ITS#8062 also handle subcursors 2015-04-16 00:20:28 +01:00
Howard Chu
b0fca5b956 ITS#8062 fix rebalance
(Probably fixes the ITS, definitely fixes a bug) when collapsing
the root page, fixups of other cursors was incomplete.
2015-04-15 23:20:55 +01:00
Hallvard Furuseth
2f6877ff7b Cleanup mtest* and sample-*.
Fix mdb_txn_begin(&read-only txn) calls. Test mdb_env_set_maxreaders().
Rename DBI open/close functions. Move mdb_dbi_close() out of txn.
2015-04-14 20:25:51 +02:00
Hallvard Furuseth
5224774f3c Reformat mdb_dbi_open() doc for clarity 2015-04-14 20:24:30 +02:00
Hallvard Furuseth
6a86f22b2a Cleanup MDB_env.me_txn0.
More fallout from 4d02c741b1.
2015-04-08 21:52:05 +02:00
Hallvard Furuseth
8adee9464f Fix per-txn MDB_NOMETASYNC, MDB_NOSYNC.
Fallout from 4d02c741b1. The flags
were ignored: mdb_txn_renew0/begin cleared and/or did not set them.
2015-04-08 21:51:50 +02:00
Hallvard Furuseth
54516639ac Renumber MDB_TXN_* flags, simplify flag code 2015-04-08 21:51:16 +02:00
Hallvard Furuseth
49bb9e82e8 mdb_txn_renew(): Clear error from previous txn 2015-04-08 21:47:24 +02:00
Howard Chu
d2dab3c905 ITS#8062 fix uninit'd cursor index 2015-03-18 19:26:21 +00:00
Howard Chu
04af46b1c1 ITS#8066 fix mdb_load with large values 2015-02-26 21:37:01 +00:00
Quanah Gibson-Mount
6fe12f3273 Happy New Year 2015-02-12 11:17:59 -06:00
Quanah Gibson-Mount
f526f1b9a3 Happy New Year 2015-02-12 10:46:33 -06:00
Howard Chu
8bb541bc40 Also support NOMETASYNC on txn_begin 2015-02-02 10:33:41 +00:00
Howard Chu
3368d1f5e2 Support MDB_NOSYNC on mdb_txn_begin() 2015-01-30 09:21:39 +00:00
Hallvard Furuseth
570ba6fb86 Fix EACCES description 2015-01-27 06:10:57 +01:00
Hallvard Furuseth
b2ab9910dd ITS#7971 Fix reader allocation and me_numreaders 2015-01-17 06:32:12 +01:00
Hallvard Furuseth
404697b369 mdb_env_set_flags(): Reject CHANGELESS flags.
Reverts part of a2ac10107e.
MDB_ROBUST needed to be accepted, but that flag is gone.
2015-01-14 08:12:50 +01:00
Howard Chu
71741a6b08 Fix potential null deref (coverity) 2015-01-13 19:19:09 +00:00
Hallvard Furuseth
d711e357da Whitespace (align with mdb.RE) 2015-01-12 21:02:29 +01:00
Hallvard Furuseth
631970e485 Make SysV semaphores robust. Cleanup MDB_ROBUST. 2015-01-12 21:02:29 +01:00
Hallvard Furuseth
9441012435 ITS#8021 doc: Don't mix MDB_WRITEMAP + non-WRITEMAP 2015-01-12 12:19:51 +01:00
Howard Chu
8b6c4250b1 More cleanup for fdatasync hack 2015-01-12 10:42:26 +00:00
Howard Chu
ea89e3d269 Tweak conditionals for fdatasync hack 2015-01-11 11:37:37 +00:00
Howard Chu
f83f62a7ae ITS#8021 env_sync is invalid in RDONLY env 2015-01-09 11:25:52 +00:00
Howard Chu
462dc09745 fdatasync hack, again
Check for ext3/ext4 fs, then check kernel version.
2015-01-08 12:54:14 +00:00
Howard Chu
e86072a9c7 Revert "Note MDB_SAFE_FDATASYNC"
This reverts commit 293d6bb47f.

Revert "Simpler fdatasync hack"

This reverts commit 9585c01233.
2015-01-08 12:17:58 +00:00
Howard Chu
293d6bb47f Note MDB_SAFE_FDATASYNC 2015-01-08 11:43:57 +00:00
Howard Chu
9585c01233 Simpler fdatasync hack 2015-01-08 11:42:08 +00:00
Howard Chu
0ef1e0b16d Revert "Fix prev commit for env_sync0"
This reverts commit 985bbbbdd5.

Revert "Hack for potential ext3/ext4 corruption issue"

This reverts commit 0018eeb2c3.
2015-01-08 11:29:30 +00:00
Howard Chu
985bbbbdd5 Fix prev commit for env_sync0 2014-12-21 00:13:50 +00:00
Howard Chu
0018eeb2c3 Hack for potential ext3/ext4 corruption issue
Use regular fsync() if we think this commit grew the DB file.
2014-12-18 04:38:53 +00:00
Howard Chu
7ce29b9edb Fix MDB_APPEND doc
We actually compare the new key against the last key, to prevent
corruption.
2014-12-15 20:40:59 +00:00
Hallvard Furuseth
6c71173258 make clean: remove *.lo 2014-12-13 20:35:08 +01:00
Hallvard Furuseth
cccc947b4a Try to avoid an invalid datafile after failed init 2014-12-08 09:56:36 +01:00
Howard Chu
9cc04f604f Clarify mdb_dbi_open doc
The concurrency restriction is on threads within a single process.
Multiple processes can use mdb_dbi_open without affecting each other.
2014-12-08 03:21:09 +00:00
Hallvard Furuseth
03ddbcf37d C90-compatible code for SysV semaphores 2014-12-05 21:00:21 +01:00
Leo Yuriev
1a7243b3f4 ITS#7987 fix excessive space for single write txn 2014-12-05 19:30:31 +00:00
Howard Chu
c36c167cc4 ITS#7994 fix prev commit
Strip docs promising behaviors that we don't promise to hold.
Strip MDB_txnid_t typedef.
2014-12-05 19:18:01 +00:00
David Barbour
b660491d3f ITS#7994 Access to current transaction ID.
I, David Barbour, hereby place the following modifications to OpenLDAP
Software (and only these modifications) into the public domain. Hence,
these modifications may be freely used and/or redistributed for any
purpose with or without attribution and/or other notice.
2014-12-05 19:17:52 +00:00
Hallvard Furuseth
1413de3a04 Fix SysV semaphores - repair mti_txnid.
Similar to 443a7e40c4.
2014-12-05 18:18:53 +01:00
Hallvard Furuseth
3e6ac6ef6b For ITS#7789: Ensure mapsize >= pages in use.
Check new mapsizes against mm_last_pg.  Move
mdb_env_init_meta0() so it can set mm_last_pg earlier.
2014-12-03 12:37:55 +01:00
Hallvard Furuseth
c306423adf Fix WIN32 -> _WIN32 2014-12-03 12:27:53 +01:00
Hallvard Furuseth
0697869d92 mdb_strerror: Silence warning @ Windows 2014-12-01 08:59:29 +01:00
Hallvard Furuseth
4376a19c5c Fix comments 2014-12-01 08:59:29 +01:00
Hallvard Furuseth
4bda1d2461 Clean up SysV semaphore code.
Change MDB_LOCK_FORMAT to differ from Posix semaphores/mutexes.
Do not use array/member length 0 for MDB_txninfo.mt2.
2014-12-01 08:59:29 +01:00
Hallvard Furuseth
443a7e40c4 Fix robust mutexes - repair mti_txnid.
Broken by e3b6c359a9, if
commit() crashes in write_meta before setting mti_txnid.
2014-12-01 08:59:29 +01:00
Hallvard Furuseth
42110d83a9 Simplify recent changes a bit. 2014-12-01 08:59:29 +01:00
Hallvard Furuseth
9a72292ac1 ITS#7961,#7987 Re-fix txn init.
More fallout from 4d02c741b1:
Don't modify another thread's write txn. Reinit me_txn0 fully.
2014-11-28 16:32:23 +00:00
leo@yuriev.ru
9a8eb95674 ITS#7971 LMDB: clarification in mdb_txn_renew0(). 2014-10-18 07:00:52 +01:00
leo@yuriev.ru
bd6d2e6f2c ITS#7970 LMDB: Critical Heisenbug
Inconsistent reading & SIGSEGV due to the race condition.
2014-10-18 06:57:58 +01:00
leo@yuriev.ru
9a80a8a8e8 ITS#7969 LMDB: volatile & __synchronize().
Globally shared fields of meta-data are not 'volatile'.
2014-10-18 06:53:53 +01:00
Howard Chu
925e7805a5 ITS#7959 fix prev commit
fstat returns -1, not an errno. Move code into its own function
for reuse.
2014-10-08 19:25:27 +01:00
Howard Chu
d72b2f5d15 ITS#7961 fix txn init
More fallout from 4d02c741b1
2014-10-04 21:48:49 +01:00
Jean-Christophe DUBOIS
d77e3ddb4d ITS#7959 Check fstat return value
Signed-off-by: Jean-Christophe DUBOIS <jcd@tribudubois.net>
2014-10-03 21:10:49 +01:00
Leo Yuriev
461d9c8f34 ITS#7841 trivial cleanup
TRIVIA - lmdb: clean testdb-dir while "make test".
2014-10-02 23:43:57 +01:00
Howard Chu
56c2c160be ITS#7956 fix compact of empty env 2014-10-02 22:11:43 +01:00
Howard Chu
bfe297da47 ITS#7955, #7671 fix MDB_PREV_DUP 2014-10-02 21:58:20 +01:00
Howard Chu
61dd017808 ITS#7953 .gitignore *.lo 2014-10-02 21:35:15 +01:00
Howard Chu
fd0315d6e5 ITS#7952 .gitignore mdb_dump/load 2014-10-02 21:34:55 +01:00
Howard Chu
8bec53fdfd Don't use -fPIC for static lib 2014-10-01 21:00:03 +01:00
Howard Chu
e3b6c359a9 Use robust mutexes by default
Making it optional on pthreads just complicates things;
they're always robust on other implementations
2014-10-01 20:58:15 +01:00
Howard Chu
58ddb5527b Use SysV semaphores instead of POSIX
Since they can cleanup after themselves on process exit
2014-10-01 20:37:25 +01:00
Howard Chu
a53716ed96 Must do dlist_free before releasing Wmutex 2014-10-01 20:36:37 +01:00
Hallvard Furuseth
ec6cf4e40f MDB_ROBUST: Use MDB_PID_T. Drop a wmutex variable. 2014-10-01 20:36:21 +01:00
Howard Chu
f6add293a7 Free write mutex earlier in txn_reset0 2014-10-01 20:36:11 +01:00
Hallvard Furuseth
a2ac10107e Add MDB_ROBUST 2014-10-01 20:35:12 +01:00
Howard Chu
a7639a66a4 Android also only has memalign 2014-09-25 19:25:56 +01:00
Howard Chu
2f587ae081 new date 2014-09-20 07:24:32 +01:00
Howard Chu
62e4eeb7f7 ITS#7943 reinit txn flags
When reusing the main write txn
2014-09-19 01:45:43 +01:00
Howard Chu
891e662754 Plug leak in 4d02c741b1
Free writer txn on env_close
2014-09-18 01:22:02 +01:00
Howard Chu
674f52fdb4 Silence some clang warnings 2014-09-16 21:54:07 +01:00
Howard Chu
b9495245b4 Release 0.9.14 2014-09-15 19:48:32 +01:00
Quanah Gibson-Mount
5ee99f1125 Change retry to num times 60. Testing shows that on a known dataset, this has the same growth behavior as 2.4.39, while num times 20 resulted in significant growth. 2014-09-13 16:21:17 -05:00
Howard Chu
3646ba966c More for me_pgoldest
Leave it in main env struct, it doesn't need to be rolled back
with nested txns.
2014-09-09 19:44:23 +01:00
Howard Chu
29fd241fad Remember oldest reader txnid
Don't walk the readers table unless we really need to.
2014-09-09 19:40:05 +01:00
Howard Chu
4d02c741b1 Use a single write txn 2014-09-04 10:40:11 +01:00
Howard Chu
a054a194e8 ITS#7913 cleanup
we already checked for data == NULL, drop unnecessary if
2014-08-24 21:18:03 +01:00
Howard Chu
8fbbc71506 More for MIPS
Move CACHEFLUSH, avoid unnecessary if()
2014-08-21 17:40:20 -07:00
Howard Chu
4008e5e583 Partially revert ada2630699
Of course, MacOSX only has posix_memalign, and only in recent versions.
(Older versions could use valloc instead. That precludes ever changing
the LMDB pagesize to be different from the OS pagesize.) Not all that
crucial since it only affects mdb_env_copyfd1.
2014-08-17 20:44:53 -07:00
Hallvard Furuseth
cf4fe3b1fb Microoptimize loose pages 2014-08-17 01:50:32 +02:00
Hallvard Furuseth
1e7eeed28d Fix loose pages in mdb_freelist_save().
Leaving them in dirty_list caused breakage.  Instead merge them
into me_pghead at the end, when no more pages will be allocated.
2014-08-17 00:29:12 +02:00
Hallvard Furuseth
983f896aa7 Split out mdb_midl_xmerge, add MDB_IDL_ALLOCLEN 2014-08-16 11:34:44 +02:00
Howard Chu
992a96e171 ITS#7917 2014-08-11 11:01:20 -07:00
Howard Chu
322cd26a7b ITS#7917 fix mdb_dbi_open
Don't let dummy go out of scope
2014-08-11 11:00:11 -07:00
Howard Chu
ada2630699 Just use memalign, not posix_memalign
Everything has it. Solaris doesn't have posix_memalign.
2014-08-07 11:06:53 -07:00
Howard Chu
899b23f89d More for MIPS
Do a cacheflush after writing data pages, separate from meta's cacheflush
2014-08-04 11:40:24 -07:00
Howard Chu
ae0c9f484d ITS#7793 more for branch key update 2014-08-04 09:26:35 -07:00
Howard Chu
9cf1749f73 ITS#7793 doc update, again 2014-08-04 05:13:29 -07:00
Howard Chu
f21f15e5a0 ITS#7793 update branch key if needed 2014-08-04 04:57:53 -07:00
Howard Chu
9febe03eb5 More for MIPS
Paranoia for 3630066843 do the
cacheflush before setting mti_txnid.
2014-08-02 10:33:35 -07:00
Howard Chu
8498c6cba2 ITS#7913 mdb_cursor_get(*_DUP) cleanup
FIRST_DUP/LAST_DUP should succeed even on non-duplicate values
PREV_DUP should reset cursor's EOF flag.
2014-07-31 09:59:52 -07:00
Howard Chu
3630066843 Fix MIPS cache coherency on Linux
MIPS chips require manual control of on-chip caches. The cacheflush
syscall being used here only exists on MIPS Linux, other OSs will
require revisiting.
2014-07-26 18:16:02 -07:00
Howard Chu
d098184f61 Update date 2014-07-24 04:15:45 -07:00
Howard Chu
082bac1492 more for mdb_cursor_count
return MDB_NOTFOUND if the cursor is empty
2014-07-24 04:13:55 -07:00
Hallvard Furuseth
6cb1166635 Fix mdb_copy & co when MDB pagesize < OS pagesize 2014-07-24 04:04:24 -07:00
Howard Chu
1b08d8e503 ITS#7789 doc updates 2014-07-23 17:31:34 -07:00
Hallvard Furuseth
81b502e17e ITS#7789 Fix mdb_env_write_meta().
Always write mapsize = max(other metapage:mapsize, env:mapsize).
2014-07-23 17:25:42 -07:00
Howard Chu
7ef970436d Revert ITS#7789 commits, let's try this again.
Revert "ITS#7789 Fix resize vs MDB_NOMETASYNC, and a comment."
Revert "ITS#7789 more"
Revert "Revert "ITS#7789 update RESIZED errmsg text""
Revert "ITS#7789 update RESIZED errmsg text"
Revert "ITS#7789 persist mapsize changes"

This reverts commit 02285aca58.
This reverts commit 38e71c68de.
This reverts commit a5e4eecb1e.
This reverts commit d8e18551c3.
This reverts commit 5926e54bba.
2014-07-23 17:19:33 -07:00
Howard Chu
015d6b909a Fix env_open mode bits 2014-07-21 18:20:31 -07:00
Howard Chu
58e9c3add8 Fix mdb_cursor_count, reject uninit'd cursor 2014-07-21 08:33:59 -07:00
Howard Chu
fcb0768e7d Silence prototype warnings 2014-07-21 08:11:33 -07:00
Howard Chu
204637b18d Typo 2014-07-14 09:53:53 -07:00
Howard Chu
9049efe493 ITS#7896 fix for mdb_env_map
Don't pass non-NULL map address unless MDB_FIXEDMAP was used.
2014-07-14 09:53:36 -07:00
Howard Chu
886ee41d55 More for cursor EOF
Must also set in sub-cursor if deleting entire subDB
2014-07-09 15:03:54 -07:00
Howard Chu
7f038d0f15 Zero-init copy2 writebuf 2014-07-09 12:05:37 -07:00
Hallvard Furuseth
2825ad64c2 MDB_MAXKEYSIZE = 0 (dynamic value) when MDB_DEVEL 2014-07-09 20:18:08 +02:00
Hallvard Furuseth
b3dedde612 MDB_DEVEL doxygen fix 2014-07-09 20:17:13 +02:00
Hallvard Furuseth
02285aca58 ITS#7789 Fix resize vs MDB_NOMETASYNC, and a comment. 2014-07-09 20:16:33 +02:00
Howard Chu
bda6a60ad4 Windows fixes
Always set the filesize when opening for writes. Otherwise
can't use backups from mdb_copy.
2014-07-09 10:22:25 -07:00
Howard Chu
f3573a7383 Windows compat
We mainly use Win32 functions, so cannot just use C-runtime strerror
to return error messages. We have to use Win32-specific messages.
Unfortunately, we document the API to return C-runtime error codes,
so we can't just switch to all Win32 error codes.
2014-07-09 10:02:21 -07:00
Howard Chu
c4d0f90fce Fix loading of printable dump
Off-by-one in end check, would complain on valid input.
2014-07-09 09:16:03 -07:00
Howard Chu
38e71c68de ITS#7789 more
persist size changes from env_open() as well. This was the
original behavior.
2014-07-09 06:54:32 -07:00
Howard Chu
a5e4eecb1e Revert "ITS#7789 update RESIZED errmsg text"
This reverts commit d8e18551c3.
And partially reverts mapsize check. Only mapsize increases are relevant.
2014-07-09 06:19:06 -07:00
Howard Chu
d5371347e8 ITS#7825 more
Set read txn's dbiseqs to env's. Set child txn's dbiseqs to parent's.
Simplify DBI_CHANGED test, no need to check md_name. No-op dbi_close
of already closed handle.
2014-07-09 05:55:30 -07:00
Howard Chu
e0273020bd Updates #7789, #7825 2014-07-08 21:40:11 -07:00
Howard Chu
276aa55969 ITS#7825 more
only update dbiseq in dbi_open on write txns
2014-07-08 21:31:48 -07:00
Howard Chu
04851e442f ITS#7825 tweak prev commit
dbiseqs are only checked in write txns so omit from read txns
2014-07-08 21:22:07 -07:00
Howard Chu
0401f2deed ITS#7825 add MDB_BAD_DBI error code
Use DBI sequence numbers to detect DBIs being closed while in use.
2014-07-08 21:02:40 -07:00
Howard Chu
d8e18551c3 ITS#7789 update RESIZED errmsg text 2014-07-08 20:05:31 -07:00
Howard Chu
5926e54bba ITS#7789 persist mapsize changes
Write decreases too, not just increases. Check for any size change
that was not requested by this process.
2014-07-08 19:18:02 -07:00
Howard Chu
604256ceff Update LMDB version date/info 2014-07-08 16:37:28 -07:00
Howard Chu
2f04728471 ITS#7892 Fix MacOSX section name 2014-07-08 16:27:03 -07:00
Hallvard Furuseth
e1a676eb18 ITS#7713 cleanups
'#ifdef MDB_DEVEL' -> '#if MDB_DEVEL'
Fix comments, use actual PAGEBASE macro as originally intended.
Actually enable 64K pages. Set MDB_DATA_VERSION for testing only.
2014-07-08 15:01:18 -07:00
Hallvard Furuseth
6ed295b256 Fixes for loose pages
mdb_txn_commit(child): Give loose pages to parent.
Use a pointer beyond the page header instead of mp_next, so
we will not need to save/restore mp_pgno. This avoids a crash
caused by references to mp_pgno.
2014-07-08 14:39:57 -07:00
Howard Chu
b3e8c71dc7 Re-fix cursor EOF
Fix in 534dcc7212 was in wrong place
2014-07-08 11:54:24 -07:00
Howard Chu
68e6faad5c ITS#7713 handle 65536 byte page size
Fix mp_lower/mp_upper to exclude PAGEHDRSZ, which means our
max mp_upper is now pagesize - PAGEHDRSZ and won't overflow.
Currently must define MDB_DEVEL to use. Will be default for
LMDB 1.0/OpenLDAP 2.5
2014-07-07 17:31:15 -07:00
Howard Chu
534dcc7212 Set xcursor EOF after deleting last entry 2014-07-07 17:13:51 -07:00
Howard Chu
1ac5147fc1 Compacting copy doesn't need to snapshot metas 2014-07-06 17:47:25 -07:00
Howard Chu
61f7ec9bfa Windows fixes 2014-07-06 07:49:38 -07:00
Howard Chu
7d996d7f3d Play games with object file layout
Use gcc section attribute to keep less frequently used functions
away from main code.
2014-07-05 08:48:47 -07:00
Howard Chu
3e98addbe2 Rationalize mdb_env_copy2 API 2014-07-05 07:49:10 -07:00
Howard Chu
059b357d1a More tweaks to copyfd2
Make sure the writer thread starts and stops when we expect it to.
2014-07-03 14:26:14 -07:00
Howard Chu
03f0ecb001 More copyfd2 tweaks 2014-07-03 10:49:05 -07:00
Howard Chu
6043103c13 Fix envinfo to return fixed map address 2014-07-03 08:31:57 -07:00
Howard Chu
64191264b8 Windows compat 2014-07-03 07:49:22 -07:00
Howard Chu
8836b78e94 Fix subDB/ovpage copying 2014-07-02 11:44:06 -07:00
Howard Chu
af3c340758 Larger WBUF still better on large DB 2014-07-02 08:07:11 -07:00
Howard Chu
342cd72f25 Don't check LEAF2 pages for subnodes 2014-07-02 07:45:12 -07:00