Commit graph

3900 commits

Author SHA1 Message Date
Howard Chu
0cccf79a02 Fix mdb_rebalance
Need to check NUMKEYS as well as fill threshold, when deciding
whether to change anything. Don't let the page drop below the
minimum number of keys.
2013-04-14 23:59:54 -07:00
Howard Chu
14c5a5dff1 Revert "ITS#7570 fix MDB_GET_MULTIPLE and MDB_NEXT_MULTIPLE"
This reverts commit 372b3ddb85.

Original code works as designed.
2013-04-14 16:53:30 -07:00
Claude Brisson
372b3ddb85 ITS#7570 fix MDB_GET_MULTIPLE and MDB_NEXT_MULTIPLE 2013-04-11 15:10:23 -07:00
Howard Chu
1da5f1faf0 Merge remote-tracking branch 'origin/mdb.master' 2013-04-07 19:23:19 -07:00
Howard Chu
fbe6a6051b Fix 227329c8e1
mdb_dbi_close() must allow any dbi < maxdbs, since opens in an
uncommitted txn don't increment env->me_numdbs.
2013-04-07 19:21:46 -07:00
Howard Chu
15af1b8cac Merge remote-tracking branch 'origin/mdb.master' 2013-04-05 03:48:05 -07:00
Howard Chu
e43fa77eaa Revert "Fix 6beaad52129da5353fd40c0ec48e6a78c4f71a2e"
This reverts commit 8eef7a4275.
Previous commit was correct, duplicate keys should not appear here.
2013-04-05 03:44:56 -07:00
Howard Chu
55d91653fd Merge remote-tracking branch 'origin/mdb.master' 2013-04-05 03:04:31 -07:00
Howard Chu
227329c8e1 Fixes for dbi_open/close, ITS#7515 2013-04-05 02:58:08 -07:00
Howard Chu
6b46799379 ITS#7561 Fix mdb_drop bugs
Check for overflow pages, reset cursor position properly.
2013-04-04 16:52:55 -07:00
Howard Chu
8eef7a4275 Fix 6beaad5212
In APPEND don't immediately reject matching key, since this
is valid for APPENDDUP.
2013-04-04 07:25:21 -07:00
Howard Chu
6beaad5212 Fix d3990eb2f8
Append mode should *reject* keys that are too small.
Also allow APPENDDUP in mdb_put().
2013-04-03 18:26:41 -07:00
Howard Chu
e31d748dbe Fix rebalance when collapsing root page
When fixing other cursors, must also fix their depth.
2013-04-01 20:34:40 -07:00
Howard Chu
cb1cc64831 More fixes for MDB_LAST
Make sure C_INITIALIZED gets set on successful call
2013-04-01 19:49:40 -07:00
Howard Chu
3213fc4db7 ITS#7556 fix cursor_sibling for PREV 2013-04-01 19:06:05 -07:00
Hallvard Furuseth
7d643d3acb Plug MDB memory leak of DB names. 2013-03-31 23:50:02 +02:00
Hallvard Furuseth
ccb7b26916 Silence signedness warning 2013-03-31 23:48:10 +02:00
Hallvard Furuseth
a70b026f6b Fix MDB_DEBUG compile when no varargs macros.
Variables mdb_debug, mdb_debug_start were used undefined.
2013-03-31 23:47:45 +02:00
Howard Chu
30da15a18d Tweak mode_t def per Hallvard's suggestion 2013-03-31 20:56:08 +01:00
Howard Chu
0c568117ce MSVC doesn't define mode_t 2013-03-31 08:10:43 -08:00
Howard Chu
7aae15739b Fix CURSOR_LAST/CURSOR_PREV 2013-03-29 01:08:06 -07:00
Howard Chu
b107cffb87 Merge remote-tracking branch 'origin/mdb.master' 2013-03-26 11:37:14 -07:00
Howard Chu
863fa77a08 ITS#7553 better fix 2013-03-26 11:24:59 -07:00
Howard Chu
faf077bd5c Merge remote-tracking branch 'origin/mdb.master' 2013-03-25 12:16:49 -07:00
Howard Chu
8a99b00acc ITS#7553 deinit cursor after deleting last node 2013-03-25 12:15:53 -07:00
Howard Chu
bccbcd1938 Silence stupid warnings 2013-03-25 11:53:32 -07:00
Howard Chu
dc2e2dbaa1 Merge remote-tracking branch 'origin/mdb.master' 2013-03-23 14:06:33 -07:00
Claude Brisson
bbb27cde4a bugfix for mdb_cursor_put with MDB_MULTIPLE
If the variable dkey.mv_size is non-zero, then it means dkey
contains some original data which has to be put back in the
child db, typically when the child db has just been created.

But when using MDB_MULTIPLE, if this variable has not been
reset to zero, we may come back to this section and wrongly
think that there is some original data to be taken care of.
2013-03-23 14:03:12 -07:00
Howard Chu
334ca55089 Merge remote-tracking branch 'origin/mdb.master' 2013-03-12 17:34:40 -07:00
Howard Chu
77001f549b ITS#7538 fallout from ITS#7536 fix.
Allow leaf pages to have only 1 key.
2013-03-12 17:29:34 -07:00
Howard Chu
3247d7cbd6 Precalculate max size of a node 2013-03-12 17:25:49 -07:00
Howard Chu
8c1d95a438 Fix f97552a83a
Causes Bus Error on SPARC
2013-03-11 10:28:25 -07:00
Howard Chu
f80171e079 Check for MADV_RANDOM
Android supports madvise but not posix_madvise
2013-03-11 06:04:54 -07:00
Howard Chu
9c4ee82297 ITS#7377 Fix c0f3d9b9a8
Add missing parens
2013-03-11 05:59:07 -07:00
Howard Chu
b1da555c4c Turn off readahead on main mmap
It's harmful when the DB is larger than RAM.
2013-03-11 05:49:14 -07:00
Howard Chu
8289ac3b5c Merge remote-tracking branch 'origin/mdb.master' 2013-03-06 15:50:47 -08:00
Howard Chu
d74330128a ITS#7536 fix mdb_rebalance
A page must always have at least 2 keys (unless it's a root page)
2013-03-06 15:10:22 -08:00
Howard Chu
6dab37f2ac Use CC/LDFLAGS to link the shared lib 2013-02-25 05:06:51 -08:00
Howard Chu
d8b407d9a2 Install manpages too 2013-02-25 02:02:15 -08:00
Howard Chu
26e4222ffb Delete obsolete @todo
We do it now... (coalescing freelist entries in page_alloc)
2013-02-24 23:43:36 -08:00
Howard Chu
58e27b6c20 Add sample apps for comparison/education purposes 2013-02-24 23:43:18 -08:00
Howard Chu
66c9f03055 Add $(DESTDIR) to make install target 2013-02-25 06:50:51 +00:00
Howard Chu
f839815778 Merge remote-tracking branch 'origin/mdb.master' 2013-02-21 05:20:20 +00:00
Howard Chu
3394bac2c0 Update error code instances 2013-02-20 05:08:52 -08:00
Hallvard Furuseth
80cd881184 Tweak MDB_INCOMPATIBLE description 2013-02-20 12:19:45 +01:00
Hallvard Furuseth
fbd76c44e4 Tweak prev commit: Restore if-test as an assert 2013-02-20 09:08:41 +01:00
Hallvard Furuseth
208e5c614d ITS#7515 Fix mdb_txn_commit(nested txn).
Don't modify the parent txn until the current txn cannot fail.
Don't assume new dirty child pgnos > dirty parent pgnos.

Page alloc/touch: Fail if child+parent dirty pages would exceed
dirty_list's maxsize. Avoids an error situation in commit.
2013-02-19 18:44:10 -08:00
Hallvard Furuseth
f43ae20be7 ITS#7512 Plug mdb_txn_abort(nested txn) page leaks.
Also catch mdb_cursor_shadow() errors.
2013-02-19 22:03:04 +01:00
Hallvard Furuseth
f97552a83a Check DB flags when refreshing a stale MDB DBI.
It's hairy to figure out when a DBI is valid. Catch destructive
user errors, and flags which another process changed under us.
2013-02-19 22:02:37 +01:00
Hallvard Furuseth
2dbb8bb833 mdb_cursor_prev,mdb_cursor_next: Fix return value.
Return mdb_node_read()'s return value if it fails, not 1.
(Can happen if mdb_page_get() fails and NDEBUG is #defined.)
2013-02-19 22:02:15 +01:00
Hallvard Furuseth
e4af9ee5da ITS#7515 mdb_dbi_open(): Also open in parent txns.
This makes aborting nested and non-nested txns more
similar: The new DBI is available to the surrounding
context (parent txn and MDB_env respectively).
2013-02-19 22:01:29 +01:00
Hallvard Furuseth
4b67270374 mdb_page_alloc(): Handle freeDB txnid range holes.
A txn writes no freeDB entry if previous txn dropped mainDB and a read
txn prevents freelist entry reuse. This surprised mdb_page_alloc (and
mdb_txn_commit too before 65c053a6e7).
2013-02-19 21:17:33 +01:00
Hallvard Furuseth
f19655eabc ITS#7517 Document that dirty DBs may not be closed 2013-02-19 21:15:26 +01:00
Hallvard Furuseth
8ad25001ff ITS#7485 Document key/data size limits in lmdb.h.
mdb.c already describes them. The user doc should too.
2013-02-19 21:14:23 +01:00
Hallvard Furuseth
ef25056cfb Revert "ITS#7515 update parent's mt_next_pgno on child commit"
This reverts commit fd4861bf00.
It duplicated earlier code.
2013-02-17 08:42:14 +01:00
Howard Chu
fd4861bf00 ITS#7515 update parent's mt_next_pgno on child commit 2013-02-17 00:49:53 +00:00
Howard Chu
890f1da3ee Don't limit retries when coalescing freelist
Try to use whatever's available.
2013-02-17 00:48:43 +00:00
Hallvard Furuseth
00d7a96bd5 ITS#7515 Fix MDB parent/child txn interaction.
mdb_txn_commit(child): Copy more state. Copy all of mt_dbs:
Include mainDB, and even freeDB since mdb_drop() can update
it. Don't skip DBs with unchanged root, this could break
when the new was newly opened and the old unused junk.

mdb_page_get(): Search parents' dirty lists.
2013-02-16 19:11:20 +01:00
Hallvard Furuseth
d90581fa5a ITS#7377 Catch MDB failure updating root pointers.
"cannot fail" was wrong, it fails at least when exceeding mapsize.
2013-02-16 19:08:54 +01:00
Hallvard Furuseth
c7db955a94 ITS#7517 Don't save dropped dirty MDB databases.
mdb_txn_commit's attempt to save the DB could corrupt another DB if
another thread had called mdb_dbi_open and reused the closed DBI.
2013-02-16 19:08:37 +01:00
Hallvard Furuseth
8e1bbdf0dd mdb_stat -ff[f]: show contiguous page spans. 2013-02-16 19:07:16 +01:00
Hallvard Furuseth
7aba5f5ab9 Add error code MDB_MAP_RESIZED. 2013-02-16 19:06:28 +01:00
Howard Chu
c6a4982793 More for ITS#7455
Don't memmove freelist entry when chunking it. Just advance the pointer,
now that it's no longer a complex struct. Also just calculate maxfree_1pg
at startup; it's a constant during runtime.
2013-02-15 13:33:50 +00:00
Hallvard Furuseth
1f953c5559 ITS#7455 Save freelist in single-page chunks 2013-02-15 13:33:50 +00:00
Hallvard Furuseth
7cacc0ec81 Freelist cleanup/streamlining
Drop unneeded definitions, redundant code.
2013-02-15 13:33:49 +00:00
Howard Chu
5e59695b8d Don't memmove freelist entry when chunking it
Just advance the pointer, now that it's no longer a complex struct
2013-02-15 13:27:50 +00:00
Hallvard Furuseth
52ecd38e18 ITS#7455 Save freelist in single-page chunks 2013-02-15 13:27:50 +00:00
Hallvard Furuseth
1b8bfc5756 Freelist cleanup/streamlining
Drop unneeded definitions, redundant code.
2013-02-15 13:27:02 +00:00
Howard Chu
9ac070beb3 Merge remote-tracking branch 'origin/mdb.master' 2013-02-07 18:30:58 +00:00
Hallvard Furuseth
65c053a6e7 ITS#7512 Fix MDB page leak when malloc error.
mdb_page_alloc(): Delay moving me_pgfirst,me_pglast
until malloc(MDB_oldpages to hold the IDs) succeeds.
2013-02-07 08:17:30 +01:00
Howard Chu
e0eb495fb3 Revert "ITS#7515 fix mdb_dbi_open/close"
This reverts commit 48dc782ea6.
2013-02-04 10:53:17 -08:00
Howard Chu
744e311939 NULL key is allowed for MDB_CURRENT 2013-02-01 06:19:23 -08:00
Howard Chu
48dc782ea6 ITS#7515 fix mdb_dbi_open/close
If a DBI handle is opened by a txn that aborts, the DBI handle
should no longer be valid.
2013-01-31 20:05:48 -08:00
Quanah Gibson-Mount
bc4e47a5a4 Merge remote branch 'origin/mdb.master' 2013-01-31 16:46:55 -08:00
Howard Chu
27aaecc744 ITS#7511 make sure cursor is marked valid after successful put 2013-01-28 18:45:36 -08:00
Howard Chu
9fa0c3bf31 ITS#7505 fix prev commit, partial revert 2013-01-29 01:08:24 +00:00
Howard Chu
0b8ac92b7a ITS#7505 Fix mdb_update_key when key is too big 2013-01-27 18:02:18 +00:00
Howard Chu
1c581612f7 Merge remote-tracking branch 'origin/mdb.master' 2013-01-26 18:56:29 +00:00
Howard Chu
cfedb365b4 Fix MDB_CURRENT updating for dupsort items 2013-01-26 18:55:35 +00:00
Howard Chu
461db2de1a ITS#7497 fix lineno overflow in ldif_read_record() 2013-01-23 02:19:02 +00:00
Hallvard Furuseth
7fdf672041 Update MDB comments: Caveats, flags, etc. 2013-01-16 18:42:57 +01:00
Howard Chu
04ea399c7c Merge remote branch 'origin/mdb.master' 2013-01-15 04:29:22 -08:00
Howard Chu
8e7bb2042b ITS#7485 data sizes limited to 32 bits
That's all we have space for in a node record.
2013-01-15 04:28:47 -08:00
Howard Chu
b3345a21e0 Merge remote-tracking branch 'origin/mdb.master' 2013-01-14 16:51:37 -08:00
Howard Chu
1a0d02c4c4 ITS#7485 more size checks 2013-01-14 13:46:25 -08:00
Howard Chu
25a99d4024 ITS#7485 check maxkeysize in mdb_cursor_put
Also MDB_MAXKEYSIZE is redefinable at compile time.
2013-01-14 13:12:15 -08:00
Howard Chu
9e35127f1d One more rename
Forgot this in 98fe339089
2013-01-14 12:57:54 -08:00
Howard Chu
53cf2eed90 ITS#7491 check for filled dirty page list
Very large single transactions will fail. It's not just a problem when
nested transactions are used. We could make this dynamically sized,
but I'm not sure what the point is.
2013-01-11 11:45:25 -08:00
Howard Chu
e250e052c3 Merge remote-tracking branch 'origin/mdb.master' 2013-01-10 02:59:39 -08:00
Howard Chu
fed573cb86 Happy New Year 2013-01-10 02:58:55 -08:00
Howard Chu
8102d9d2df Merge remote-tracking branch 'origin/mdb.master' 2013-01-09 12:22:41 -08:00
Hallvard Furuseth
cd80a15474 Delete liblmdb/mfree. It has moved into mdb_stat. 2013-01-09 20:49:43 +01:00
David Bender
8f52aa24d3 ITS#7476 Prevent EINTR from stopping otherwise successful connect 2013-01-09 09:11:35 -08:00
Howard Chu
0521a7b374 Merge remote branch 'origin/mdb.master' 2013-01-09 08:57:53 -08:00
Howard Chu
2a36346b50 ITS#7455 don't give up so soon
If we're operating on the freelist, see if our current pghead
can satisfy the request before giving up and using new pages.
2013-01-09 08:30:13 -08:00
Kurt Zeilenga
0fd1bf30b8 Happy New Year 2013-01-02 10:22:57 -08:00
Howard Chu
5e1dbc48ab Merge remote-tracking branch 'origin/mdb.master' 2012-12-19 13:20:50 -08:00
Howard Chu
057e068630 Fix typo 2012-12-19 13:20:24 -08:00
Howard Chu
9f2c72e5fb Merge remote-tracking branch 'origin/mdb.master' 2012-12-19 13:12:50 -08:00
Howard Chu
460a05bf78 Add initial mdb utility manpages 2012-12-19 13:09:56 -08:00
Howard Chu
2565e974b9 ITS#7477 check for invalid LDIF 2012-12-19 09:15:09 -08:00
Howard Chu
9ed5ca4f7a Merge remote-tracking branch 'origin/mdb.master' 2012-12-11 22:22:52 -08:00
Howard Chu
82b97a8af0 init_meta should write mapsize
Was leaving it 0 initially.
2012-12-11 22:21:52 -08:00
Howard Chu
1c1b5ca0c4 Merge remote-tracking branch 'origin/mdb.master' 2012-12-11 16:01:27 -08:00
Howard Chu
0702fbd688 Update version date 2012-12-11 16:01:25 -08:00
Howard Chu
bae86317e0 Merge remote-tracking branch 'origin/mdb.master' 2012-12-11 14:22:31 -08:00
Howard Chu
9acf801cc1 Make sure init_meta is at beginning of file 2012-12-11 12:55:47 -08:00
Howard Chu
ed86a086f7 More for prev commit
Use the latest meta page
2012-12-11 12:46:17 -08:00
Howard Chu
e74d70e88e Partially revert 65d40eb5d2
Allow both increasing and decreasing the environment size.
But don't allow decreasing below the currently occupied space.
2012-12-11 12:03:19 -08:00
Howard Chu
470b126e9f Merge remote-tracking branch 'origin/mdb.master' 2012-12-10 17:22:26 -08:00
Howard Chu
282be11654 FIx prev commit 2012-12-10 17:21:24 -08:00
Howard Chu
3bf87500a3 Merge remote-tracking branch 'origin/mdb.master' 2012-12-10 12:22:30 -08:00
Howard Chu
0ce6bb4be0 Add error checks to txn_commit
These ops could fail if the map is full.
2012-12-10 12:21:36 -08:00
Howard Chu
aff2693fc0 ITS#7455 simplify
Don't try to reclaim overflow pages while operating on
the freelist (for now). The circular dependencies are much like
the single-page case, but worse. Maybe look into this in the
future, but it's not absolutely necessary now.
2012-12-10 12:16:50 -08:00
Howard Chu
e26fc73417 ITS#7467 add verbose freelist info 2012-12-07 13:19:36 -08:00
Howard Chu
6a9476822d Merge remote-tracking branch 'origin/mdb.master' 2012-12-07 12:28:25 -08:00
Howard Chu
784757faa9 ITS#7465 fail if both -a and -s specified 2012-12-07 12:06:30 -08:00
Howard Chu
6062cd9777 Merge remote-tracking branch 'origin/mdb.master' 2012-12-06 17:41:14 -08:00
Howard Chu
9a5d4370be Yet more retries for overflow page scan 2012-12-06 17:39:49 -08:00
Howard Chu
8ab758c9cc Merge remote-tracking branch 'origin/mdb.master' 2012-12-06 17:11:29 -08:00
Howard Chu
5863d5cc61 Freelist fixes
Keep list sorted if it grows during a write. Don't free pghead until
we're sure our writes are all finished.
2012-12-06 09:05:30 -08:00
Howard Chu
a037e27fc0 Merge remote branch 'origin/mdb.master' 2012-12-04 06:39:01 -08:00
Howard Chu
7b10fdde00 Fix mdb_env_copy
Just write in 2^31 sized chunks
2012-12-04 06:30:22 -08:00
Howard Chu
bbfe587246 Add support for 64 bit index hashing 2012-12-03 18:14:28 -08:00
Howard Chu
f9d41ba98a Merge remote-tracking branch 'origin/mdb.master' 2012-12-03 16:54:33 -08:00
Howard Chu
04b7c42109 Rename mdb_open/close mdb_dbi_open/close
Avoid possible symbol clashes with MDB-Tools libmdb
2012-12-03 16:53:24 -08:00
Howard Chu
e545b742e5 Fix 5c1ee7f7ba
mdb_cursor_sibling() no longer pops cursor before returning.
2012-12-03 13:57:46 -08:00
Howard Chu
5af47adf6b Fix mdb_midl_shrink 2012-12-03 13:57:28 -08:00
Howard Chu
ef3b4213ff Merge remote-tracking branch 'origin/mdb.master' 2012-11-30 12:56:58 -08:00
Howard Chu
d3f51fdcf7 More retries for overflow page scan 2012-11-30 12:54:19 -08:00
Howard Chu
1509d75363 Merge remote-tracking branch 'origin/mdb.master' 2012-11-30 12:47:00 -08:00
Howard Chu
98fe339089 Rename libmdb to liblmdb 2012-11-30 12:45:34 -08:00
Howard Chu
2a94ebced6 Add install target 2012-11-30 12:39:53 -08:00
Howard Chu
d0e86d485d Merge remote-tracking branch 'origin/mdb.master' 2012-11-30 12:33:18 -08:00
Howard Chu
13f3bcd59c Rename libmdb to liblmdb, lmdb.h
Avoid naming conflicts with other mdb* packages. Bump version to 0.9.5
2012-11-30 12:30:51 -08:00
Howard Chu
f165507199 ITS#7455 use freelist for overflow pages 2012-11-30 12:18:27 -08:00
Howard Chu
185136397d Drop unneeded #include <time.h> 2012-11-28 15:34:44 -08:00
Howard Chu
049b145ec7 Doc layout tweaks 2012-11-28 15:34:19 -08:00
Hallvard Furuseth
97ed96206d mdb_env_sync(,force=1): Override MDB_MAPASYNC.
With MDB_MAPASYNC, the API provided no way to ensure full sync.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
b9275c9991 mdb_cursor_dbi(): Assume valid args.
It returned 0, which is the freelist DBI and is valid for a few uses.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
7e620a34a2 API change: mdb_drop() only accepts del=0 or 1.
This allows for other flag values in the future.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
d903bed218 API change: mdb_env_set_maxdbs(, *named* DBs).
Let the user specify the number of databases he will actually
use. Adding 2 for FREE_DBI and MAIN_DBI is an internal matter.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
6b56e0fab1 ITS#7453 Fix reset of MDB_NOSYNC/MDB_NOMETASYNC.
mdb_env_open() with these flags did not open a synchronizing
file descriptor.  Thus disabling them later did not work.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
0201b9002c Tweak MDB_FIXEDMAP handling.
Drop mmap()'s MAP_FIXED flag, so it returns another address instead
of unmapping existing overlapping pages. Verify the returned address.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
52e3adbdec Renumber MDB_NOSUBDIR. Check mdb_env_open() flags.
MDB_NOSUBDIR was == MDB_REVERSEKEY. That affected the freelist:
Env flags are stored in mm_flags alias mm_dbs[FREE_DBI].md_flags.

It stays a persistent flag, in case mdb_env_open someday wants
to pick/verify which lockfile to use with the datafile.

Catch bad flags so they will no longer make it into the data file.
2012-11-28 00:59:00 +01:00
Ralf Haferkamp
c728ebf586 ITS#7428 Use non-blocking IO during SSL Handshake
If a timeout is set, perform the SSL Handshake using non-blocking IO.  This way
we can timeout if SSL Handshake gets stuck for whatever reason.

This code is currently hidden behind #ifdefs (LDAP_USE_NON_BLOCKING_TLS) and
disabled by default as there seem to be some problems using NON-blocking
I/O during the TLS Handshake when linking against NSS (either a bug in NSS
itself of in tls_m.c, see discussion on -devel)

This patch adds an additional parameter to ldap_int_poll() in order to indicate
if we're waiting in order to perform a read or write operation.
2012-11-21 14:25:18 +01:00
Howard Chu
88fd28886d Merge remote-tracking branch 'origin/mdb.master' 2012-11-05 09:47:03 -08:00
Howard Chu
bb36bdcd1c ITS#7432 fix typo in mdb_midl_sort
Wasn't pushing the optimal half of the array onto the stack,
thus used more stack than expected -> overrun.
2012-11-05 05:06:06 -08:00
Howard Chu
64b164b4fc Merge remote-tracking branch 'origin/mdb.master' 2012-10-30 16:34:14 -07:00
Howard Chu
0108327c27 Better fix for MDB_DEBUG tweak -- from hbf 2012-10-25 07:03:25 -07:00
Howard Chu
b68aa3f86b Revert "MDB_DEBUG tweak for old compilers"
This reverts commit ef2092eac9.
2012-10-25 06:54:09 -07:00
Howard Chu
ef2092eac9 MDB_DEBUG tweak for old compilers 2012-10-25 05:28:55 -07:00
Howard Chu
52a0b4f83c Merge remote-tracking branch 'origin/mdb.master' 2012-10-23 04:49:09 -07:00
Howard Chu
c2f15990c7 Fix flag checks in mdb_open 2012-10-23 04:48:04 -07:00
Howard Chu
f5b8d017df fix Windows mdb_env_copy(), WriteFile overflow 2012-10-23 04:32:55 -07:00
Howard Chu
f51efd4b03 Merge remote-tracking branch 'origin/mdb.master' 2012-10-22 17:05:26 -07:00
Howard Chu
66f2b7b2ec Make sure mdb_open flags are committed on main DB
If no data was modified in the txn, mdb_open flag changes were dropped.
2012-10-22 17:03:15 -07:00
Howard Chu
5f2ddb5532 Minor cleanup 2012-10-22 16:13:42 -07:00
Howard Chu
5c1ee7f7ba Fix MDB_PREV
Could return 1 garbage record before actual data, if starting from EOF
2012-10-22 16:02:30 -07:00
Howard Chu
617769bce5 Tweak output, add free page info 2012-10-16 17:01:16 -07:00
Howard Chu
754cd88819 Add mdb_env_info() to retrieve interesting env params 2012-10-16 16:22:21 -07:00
Howard Chu
7f0771a3ed Merge mdb_stata into mdb_stat 2012-10-16 15:37:29 -07:00
Howard Chu
65d40eb5d2 Make sure increases of mapsize are persisted 2012-10-16 14:53:25 -07:00
Hallvard Furuseth
94965307c8 ITS#7377 Errorcheck Windows calls and thread keys. 2012-10-15 17:04:48 +02:00
Hallvard Furuseth
88b7bb4639 Explicitly unlock the MDB lockfile on Windows. 2012-10-15 17:04:43 +02:00
Hallvard Furuseth
e9d87866c4 mdb_env_open(): Keep mdb_env_set_flags() flags. 2012-10-15 17:04:11 +02:00
Hallvard Furuseth
092f9a5c66 More for mdb_env_copy().
Fix #define _GNU_SOURCE = 1 to match any command line -D_GNU_SOURCE.
.gitignore += mdb_copy.
2012-10-15 17:03:46 +02:00
Hallvard Furuseth
462d9dfd10 Fix last commit.
Add flag MDB_ENV_ACTIVE instead of using the state of me_lfd.
Require MDB_RDONLY for MDB_ROFS.  Swap reader/writer mutex lock.
2012-10-15 17:03:33 +02:00
Howard Chu
e40713b631 Support read access on read-only media 2012-10-13 04:05:00 -07:00
Howard Chu
f037dd2d19 Silence warning in prev commit 2012-10-11 16:47:56 -07:00
Howard Chu
ebb6859ea5 Add mdb_copy for backing up a DB environment 2012-10-11 12:24:01 -07:00
Hallvard Furuseth
756ce8e10c Tweak MDB restrictions 2012-10-03 21:13:21 +02:00
Hallvard Furuseth
a3f33dd20e Fix mdb_open() off-by-one error in maxdbs check 2012-10-03 18:08:56 +02:00
Hallvard Furuseth
38fb8e6eb1 ITS#7377 Wrap fcntl F_SETLK in EINTR loop too.
This can happen even on local filesystems.
2012-10-03 18:06:29 +02:00
Hallvard Furuseth
ab3fea51dc ITS#7377 Catch ftruncate() error 2012-10-03 18:06:29 +02:00
Hallvard Furuseth
2b960613e0 ITS#7377 Document caveats/troubeshooting. 2012-10-03 18:04:07 +02:00
Howard Chu
0fc0ccdc0c Merge remote-tracking branch 'origin/mdb.master' 2012-09-30 10:03:21 -07:00
Howard Chu
7333b6bdc9 Fix prev commit
It only worked if the freelist was already non-empty
2012-09-30 10:01:57 -07:00
Howard Chu
b0ad9e76c7 Merge remote-tracking branch 'origin/mdb.master' 2012-09-30 07:50:56 -07:00
Howard Chu
cd228b4722 Fix mdb_drop
if it was the only operation in a txn, it would be ignored.
2012-09-30 07:49:30 -07:00
Howard Chu
a1c2dc66db Merge remote-tracking branch 'origin/mdb.master' 2012-09-27 05:07:24 -07:00
Howard Chu
0c4c6fe72a ITS#7385, check further for bad splits
More for 48ef27b6f5
Our definition of "large" data item needs to be smaller to
handle more cases where a bad split is possible
2012-09-27 05:06:23 -07:00
Howard Chu
2a68553ec1 Merge remote-tracking branch 'origin/mdb.master' 2012-09-19 06:16:36 -07:00
Howard Chu
ba6c3281a1 Add MDB_GET_CURRENT
return whatever the cursor is currently pointing at
2012-09-19 06:15:09 -07:00
Howard Chu
c264d70fbe Merge remote-tracking branch 'origin/mdb.master' 2012-09-18 12:37:59 -07:00
Howard Chu
d13ca0de4f ITS#7394 ignore WRITEMAP if RDONLY 2012-09-18 12:36:54 -07:00
Howard Chu
b8e48e95fb Merge remote-tracking branch 'origin/mdb.master' 2012-09-17 08:21:20 -07:00
Howard Chu
0a359fb629 More for 48ef27b6f5
page_split with newindex > split_indx
2012-09-17 07:02:41 -07:00
Howard Chu
5ef56b437e More for ab04c50a32
Use explicit MDB_RDONLY flag
2012-09-17 07:01:28 -07:00
Hallvard Furuseth
c67ea9c060 ITS#7377 Catch MDB user errors. 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
c0f3d9b9a8 ITS#7377 Catch MDB setup errors and clean up. 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
fe1b3794de ITS#7364 Limit mdb lock upgrade before sem_unlink.
Do not try shared lock when closing or after error.  Track file lock
state to decide.  Change meaning of *excl to reflect file lock state.
2012-09-17 15:42:15 +02:00
Hallvard Furuseth
c760e536ec ITS#7364 Always sem_unlink() in mdb_env_open().
Drop the sem_unlink() error checks, which could prevent the 2nd
unlink.  Instead use O_EXCL in sem_open().

This makes "open+close the database" the API for trying to clean
away the old semaphores, if they were left behind by a previous run.
2012-09-17 15:42:15 +02:00
Hallvard Furuseth
31be24896b ITS#7377 Wrap sem_wait & file locks in EINTR loops 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
a35f9b2a53 Remove mdb data races. Use (txnid_t)-1 as "no ID".
Avoid race between numreaders++ and reading numreaders at cleanup. Make
the un-mutexed reset of reader table entry, atomic: Reset mr_pid only.

Instead check mr_pid != 0 in mdb_page_alloc()'s scan for readers.
(txnid_t)-1 as "no ID"-mark avoids a check for mr_txnid != 0.
The scan can stop when seeing an old reader.
2012-09-17 15:42:14 +02:00
Hallvard Furuseth
38cc1e96b4 Save pid in MDB_env instead of repeating getpid().
An open MDB environment does not survive or catch fork(),
so repeating getpid() was pointless.
2012-09-17 15:42:14 +02:00
Hallvard Furuseth
20a216fcc9 MDB_WRITEMAP needs no DSYNC descriptor (me_mfd) 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
588a84a5ae ITS#7363 Preprocessor namespace cleanup.
Rename USE_POSIX_SEM to MDB_USE_POSIX_SEM.
Separate MDB_FDATASYNC from MDB_USE_POSIX_SEM.
2012-09-17 15:42:14 +02:00
Howard Chu
8bb10add24 More for ab04c50a32
Fix typos, error code ranges
2012-09-17 06:35:03 -07:00
Howard Chu
acbff5b1ea Add mdb_cursor_renew()
Allow cursors on read-only txns to be reused with later txns.
2012-09-17 04:41:13 -07:00
Howard Chu
076b2b36a9 Shared lib should depend on pthread 2012-09-17 03:48:54 -07:00
Howard Chu
c542442c9f Add MDB_SET_KEY cursor op
Overwrites the passed in key with the DB's key
2012-09-17 02:17:25 -07:00
Howard Chu
ad79308d5a Merge remote-tracking branch 'origin/mdb.master' 2012-09-14 06:52:37 -07:00
Howard Chu
ab04c50a32 ITS#7383 More explicit error codes 2012-09-14 06:51:31 -07:00
Howard Chu
c2bd7e630d Merge remote-tracking branch 'origin/mdb.master' 2012-09-12 10:47:36 -07:00
Howard Chu
48ef27b6f5 ITS#7385 fix mdb_page_split (again)
More for d793594173 ITS#7229
2012-09-12 10:45:51 -07:00
Howard Chu
c56ede7781 Merge remote-tracking branch 'origin/mdb.master' 2012-09-07 13:42:52 -07:00
Howard Chu
de77882898 Update comments, bump version
Note read-write vs read-only mmap tradeoffs.
2012-09-07 13:40:07 -07:00
Howard Chu
2e677bcb99 msync should default to fully synchronous 2012-09-06 11:17:25 -07:00
Howard Chu
1c733bdc4e Add msync support for WRITEMAP option 2012-09-05 08:40:17 -07:00
Howard Chu
d8a89464cd Add MDB_WRITEMAP to use a writable mmap 2012-09-05 08:39:46 -07:00
Howard Chu
14a08f877f More for 57b5fbea49
Fix abort of child txn
2012-09-04 06:02:26 -07:00
Hallvard Furuseth
4608cd9552 libmdb: Fix P_INVALID when pgno_t != unsigned long 2012-08-29 06:43:35 +02:00
Hallvard Furuseth
3e3c785034 libmdb: Check sem_open() against SEM_FAILED. 2012-08-29 06:42:52 +02:00
Howard Chu
d7dd1bdb82 Merge remote-tracking branch 'origin/mdb.master' 2012-08-27 14:45:58 -07:00
Howard Chu
4bc18a5dd3 In mdb_audit, change assert to fprintf 2012-08-27 14:45:16 -07:00
Howard Chu
85ae5011e7 Fix mdb_audit()
due to a0993354a6
2012-08-26 11:05:45 -07:00
Howard Chu
f980c64cb6 Merge remote-tracking branch 'origin/mdb.master' 2012-08-24 05:34:29 -07:00
Ondrej Kuznik
bd92b570a8 ITS#7369 mdb_stat: cleanup in case something went wrong. 2012-08-24 05:33:46 -07:00
Howard Chu
a2d1b42e39 ITS#7369 abort txn before exiting 2012-08-24 05:23:55 -07:00
Howard Chu
67dd10ed2c Merge remote-tracking branch 'origin/mdb.master' 2012-08-22 23:02:48 -07:00
Howard Chu
916995236e ITS#7364 also close the mutexes as needed
Otherwise a process that opens and closes an env more than once
will leak mutex handles. (WIN32/BSD)
2012-08-22 23:01:19 -07:00
Chris Mikkelson
46cf442700 ITS#7364 Remove POSIX semaphores when the last user closes the mdb environment. 2012-08-22 22:55:01 -07:00
Rich Megginson
a0e48e7246 ITS#7360 accept nss certname in the form of tokenname:certnickname
There are cases where the user may want to force the use of a particular
PKCS11 device to use for a given certificate.  Allow the user to do this
with MozNSS by specifying the cert as "tokenname:certnickname" where
token name is the name of a token/slot in a PKCS11 device and certnickname
is the nickname of a certificate on that device.
2012-08-22 14:21:23 -07:00
Howard Chu
1ebf95c31b ITS#7359 cleanup for loop 2012-08-22 14:13:10 -07:00
Jan Vcelak
6833b8717a ITS#7359 MozNSS: fix whitespaces in all my changes
To be compliant with OpenLDAP coding standards.
2012-08-22 14:12:01 -07:00
Howard Chu
d377f353b7 Merge remote-tracking branch 'origin/mdb.master' 2012-08-22 14:09:56 -07:00
Chris Mikkelson
9f983b7999 ITS#7363 Use posix semaphores on apple and bsd systems. 2012-08-22 14:09:08 -07:00
Jan Vcelak
46dc6c424b ITS#7359 MozNSS: prefer unlocked slot when getting private key 2012-08-21 13:32:34 -07:00
Howard Chu
68c3cf9795 ITS#7358 fix ITS#7270 patch
commit 8bb9e88d5f breaks Visual C
(as well as our own coding rules)
2012-08-21 13:22:19 -07:00
Howard Chu
a787eb641f Merge commit 'd53644f6' 2012-08-21 12:18:10 -07:00
Howard Chu
f114fec545 Loop thru all DBs and show their stats
This is quick and dirty, should be integrated into mdb_stat.
2012-08-16 10:25:40 -07:00
Howard Chu
d53644f682 Minor cleanup 2012-08-02 16:03:10 -07:00
Howard Chu
d1120fbccc Write optimization: avoid copying unused portions of pages. 2012-07-31 16:03:48 -07:00
Jan Vcelak
f425a07d02 ITS#7331 MozNSS: ignore untrusted issuer error when verifying server cert
Untrusted issuer error can appear with self-signed PEM certificates.
2012-07-26 10:16:39 -07:00
Howard Chu
8fe328dd51 Merge remote-tracking branch 'origin/mdb.master' 2012-07-26 08:43:57 -07:00
Howard Chu
234cd9dfb5 Fix txn_commit error check 2012-07-26 08:42:41 -07:00
Howard Chu
0e3fa1ee14 Merge remote-tracking branch 'origin/mdb.master' 2012-07-26 07:25:07 -07:00
Howard Chu
23219ed32a Fix 205bb17793
Ignore current cursor pos if it's out of bounds.
2012-07-26 07:22:52 -07:00
Howard Chu
d1a7fa267b Update sub-DBs before updating free list 2012-07-26 06:25:31 -07:00
Howard Chu
4317c7ce54 Merge remote-tracking branch 'origin/mdb.master' 2012-07-25 12:32:28 -07:00
Howard Chu
d3990eb2f8 Append mode should also detect keys that are too small 2012-07-25 12:31:08 -07:00
Howard Chu
cf74dfbb2d Merge remote-tracking branch 'origin/mdb.master' 2012-07-24 10:18:40 -07:00
Howard Chu
5da67968af Fix root split check in mdb_page_split 2012-07-24 08:50:42 -07:00
Mat Booth
e6d190c7de ITS#7332 Changes required to build with Microsoft Visual Studio 2012-07-23 08:29:39 -07:00
Howard Chu
205bb17793 Skip unneeded calls to mdb_page_get in mdb_page_search 2012-07-23 04:23:49 -07:00
Howard Chu
832f7de6ae Merge remote-tracking branch 'origin/mdb.master' 2012-07-22 17:58:39 -07:00
Howard Chu
ccc4d23e74 Leave extra space when creating a fake page
Minimizes the number of memcpy's if additional dups are stored.
2012-07-22 17:45:50 -07:00
Howard Chu
05ec98d119 Append mode still needs to detect duplicates 2012-07-22 17:45:50 -07:00
Emily Backes
c453a236fc Update name information 2012-07-22 07:08:35 -07:00
Howard Chu
2b1b64f021 Merge remote-tracking branch 'origin/mdb.master' 2012-07-21 06:21:36 -07:00
Howard Chu
14fb1f59c7 Append tweaks, page_split fixes
Append mode now does no key comparisons, input must be in sorted order.
page_split was not updating cursor parents correctly.
2012-07-21 06:19:09 -07:00
Howard Chu
accd19edbc Merge remote-tracking branch 'origin/mdb.master' 2012-07-18 14:19:19 -07:00
Howard Chu
7e9a6134fd Make sure cursor's DB is init'd if STALE. 2012-07-17 04:04:52 -07:00
Howard Chu
a0993354a6 Don't use env-private copy of DB root nodes.
Just lookup the DB roots as needed. When many DBs are in use,
most of the copies won't be referenced in a given txn, and
there's a bad race condition in the copy routine.
2012-07-13 15:48:26 -07:00
Howard Chu
0ea56294f1 Fix darwin sem_open() names
Must begin with '/'
2012-07-13 14:14:36 -07:00
Howard Chu
1a9775dc5a Fix ID -> MDB_ID due to 20baad4a20 2012-07-13 14:14:36 -07:00
Howard Chu
248fd1c7b8 Windows thread callback support 2012-07-13 14:14:36 -07:00
Howard Chu
43d47c46a7 ITS#6262 fix gettime() regression
Add a mutex in ldap_pvt_gettime(), delete the mutex comment
since it's no longer relevant (and was ignored anyway). This
could only ever affect multi-processor machines.
2012-07-11 16:53:33 -07:00
Howard Chu
113538806f Merge remote-tracking branch 'origin/mdb.master' 2012-07-09 08:44:12 -07:00
Howard Chu
e562f53e10 Drop me_toggle, unused 2012-07-08 18:43:45 -07:00
Howard Chu
179c44e7da Revert "libmdb: Do not use the MDB_txbody.mtb_txnid field."
This reverts commit 4a0b30499c.
2012-07-08 18:18:12 -07:00
Howard Chu
c6b5abbfd2 Merge remote-tracking branch 'origin/mdb.master' 2012-07-06 18:03:06 -07:00
Howard Chu
433105f09b Fix: avoid direct reference to meta pages
Relevant info should be copied during txn_begin only.
2012-07-06 17:38:03 -07:00
Howard Chu
df7ddb6bf4 Add MDB_NOMETASYNC env option.
Just a trial. This may not make sense if we decide to split the
meta pages into their own separate file, to allow meta traffic to
reside on a separate spindle.
2012-07-05 18:11:18 -07:00
Howard Chu
38560c2517 Tweak b802bcf704
Clarify prev commit, fix Doxygen comments broken by earlier changes
2012-07-05 17:43:29 -07:00
Howard Chu
b802bcf704 Partial revert of 13c663f666
Don't re-use free pages so soon; that leaves us vulnerable to
DB corruption if data syncs successfully but meta doesn't.
2012-07-05 16:52:13 -07:00
Jan Vcelak
2db5195650 ITS#7316 MozNSS: do not retry when reading the pin from file
Avoid infinite loop if the pin in the password file is wrong.
2012-06-26 06:47:48 -07:00
Jan Vcelak
a171237999 ITS#7315 MozNSS: do not authenticate to a slot manually
We cannot rely on tc_certificate->slot value, the library will choose
the correct slot itself when needed. Just provide correct pin argument.
2012-06-26 06:46:29 -07:00
Jan Vcelak
3531c344e1 ITS#7314 MozNSS: lock whole init and clenaup process 2012-06-26 06:44:46 -07:00
Jan Vcelak
d07779e97b ITS#7313 MozNSS: store certificate object instead of nickname in in ctx
PEM certificates should not be referenced by nicknames, because the
nicknames are derived from basename of the cerificate file and in
general are not easy-predictable.

The code of Mozilla NSS backend depends on some aspects of PEM module
and tries to guess the nicknames correctly. In some cases the guessing
is wrong.
2012-06-26 06:43:11 -07:00
Jan Vcelak
87132b820d ITS#7312 MozNSS: context specific token description for certdb 2012-06-26 06:41:05 -07:00
Jan Vcelak
221531b31a ITS#7241 unbind: free socket prior to destroying TLS context 2012-06-25 12:53:29 -07:00
Howard Chu
450d224291 ITS#7293 make ldapava_free static 2012-06-23 05:41:39 -07:00
Howard Chu
e2b5aaf5dd ITS#7290 fix ldap_int_initialize() to set debug level 2012-06-07 05:15:04 -07:00
Jan Vcelak
00d0e16272 ITS#7291 MozNSS: read pin from file file can cause infinite loop
The buffer allocated for reading password file has to be initialized
with zeros, or we need to append zero at the end of the file. Otherwise
we might read unitialized memory and consider it to be a password.
2012-06-07 05:05:08 -07:00
Tim Strobell (Contractor)
4b6bd2c600 ITS#7289 MozNSS: cipher suite selection by name may be ignored 2012-06-07 05:03:15 -07:00
Jan Vcelak
dc3842fca3 ITS#7287 MozNSS: do not overwrite error in tlsm_verify_cert 2012-06-05 03:43:24 -07:00
Tim Strobell (Contractor)
2c2bb2e7ae ITS#7285 Mozilla NSS: default cipher suite always selected 2012-06-05 03:43:15 -07:00
Quanah Gibson-Mount
3f46f2e0bc Merge remote branch 'origin/mdb.master' 2012-06-04 19:41:58 -07:00
Howard Chu
a334257800 Tweak prev commit, use first unused slot 2012-06-02 05:26:31 -07:00
Howard Chu
4cabb24b83 Fix mdb_open, re-use closed dbi slots 2012-06-02 05:24:47 -07:00
Jan Vcelak
df1d118de8 ITS#7276 MozNSS: allow CA certdb together with PEM CA bundle file
Prior to this patch, if TLS_CACERTDIR was set to Mozilla NSS certificate
database and TLS_CACERT was set to a PEM bundle file with CA
certificates, the PEM file content was not loaded.

With this patch and the same settings, OpenLDAP can verify certificates
which are signed by CAs stored both in certdb and PEM bundle file.
2012-05-30 06:18:50 -07:00
Frederik Deweerdt
8bb9e88d5f ITS#7270 Protect accesses to ldap_int_hostname with a mutex.
Not protecting the accesses to ldap_int_hostname could lead to a double
free.
2012-05-30 05:49:53 -07:00
Howard Chu
22bf5188a9 Merge remote-tracking branch 'origin/mdb.master' 2012-05-09 12:13:05 -07:00
Howard Chu
eeec509150 ITS#7266 fix rebalance for LEAF2 pages 2012-05-09 11:24:25 -07:00
Howard Chu
ff1839ebe6 Merge remote-tracking branch 'origin/mdb.master' 2012-05-08 22:19:05 -07:00
Howard Chu
a540597dd6 More freelist tweaks, avoid referencing freed page 2012-05-08 22:17:17 -07:00
Howard Chu
326d07d8c6 Merge remote-tracking branch 'origin/mdb.master' 2012-05-05 19:42:30 -07:00
Howard Chu
fadf75a9cb Further clarification of mdb_open() 2012-05-05 19:41:45 -07:00
Howard Chu
4d0d12db27 Merge remote-tracking branch 'origin/mdb.master' 2012-04-18 16:05:16 -07:00
Howard Chu
fbf9c23304 ITS#7251 fix Apple/Windows shared mutex naming 2012-04-18 15:59:54 -07:00
Hallvard Furuseth
20baad4a20 libmdb: midl.h namespace cleanup.
Prefix ID, NOID, ID2, IDL, ID2L with 'MDB_'.
Also drop bdb from file description.
2012-04-12 08:24:36 +02:00
Hallvard Furuseth
0a4aa88927 libmdb: Simplify pick_meta() from read_meta().
Put one call in #if MDB_DEBUG. Move DPRINTF(Using meta page #) there.
2012-04-12 08:15:33 +02:00
Hallvard Furuseth
dce5bb2fee libmdb: Simplify mdb_default_cmp. No real change. 2012-04-12 08:15:33 +02:00
Hallvard Furuseth
729a8a2819 libmdb: signed meets unsigned warning cleanup 2012-04-12 08:15:33 +02:00
Hallvard Furuseth
99fa22c164 libmdb: MDB_FDATASYNC code cleanup.
Define MDB_FDATASYNC() instead of redefining fdatasync().
2012-04-12 08:15:33 +02:00
Hallvard Furuseth
592b3f8727 libmdb: Add C++ 'extern "C" {}' to *.h 2012-04-12 08:15:33 +02:00
Hallvard Furuseth
8e1ebbb4a8 libmdb: Don't open datafile twice when not needed.
If the database is opened with MDB_RDONLY or MDB_NOSYNC,
instead use the same file descriptor for me_mfd and me_fd.

Also factor out Windows/Unix error handling after open.
2012-04-12 08:15:33 +02:00
Hallvard Furuseth
6d0b424dbe libmdb: Set close-on-exec flag on lockfile 2012-04-12 08:15:33 +02:00