Commit graph

1200 commits

Author SHA1 Message Date
Howard Chu
6ac9aa666e Happy New Year 2017-01-11 14:39:08 +00:00
Howard Chu
f8ce8a8271 More C_EOF tweaks 2017-01-11 11:19:18 +00:00
Howard Chu
c44b29eaa8 ITS#8557 fix prev commit 2017-01-11 11:18:57 +00:00
Howard Chu
511f5880e2 Tweak cursor_next C_EOF check
Allow C_EOF flag to be stale
2017-01-11 10:33:28 +00:00
Howard Chu
d84dee516f ITS#8557 fix mdb_cursor_last
Optimize mdb_page_search_root(PS_LAST) when cursor is already near
last position, ignoring C_EOF flag for now.
2017-01-11 09:51:43 +00:00
Howard Chu
59ac317d2a ITS#8558 fix mdb_load with escaped plaintext 2017-01-06 19:48:58 +00:00
Howard Chu
2e3eaf2ce2 ITS#8554 kFreeBSD is like BSD
Doesn't have POSIX robust mutexes - GNU userland on BSD kernel
2016-12-28 18:30:19 +00:00
Hallvard Furuseth
4bc270a2cb More MDB_node doc 2016-12-21 21:42:36 +01:00
Hallvard Furuseth
be94a7565b Doxygen fixes. Use DISTRIBUTE_GROUP_DOC.
- DISTRIBUTE_GROUP_DOC makes doxygen give several fields the
  same doc: mn_hi + mn_lo in MDB_node.
- With mdb_mutex_t + mdb_mutexref_t, instead split them up.
- Don't hide a doxygen #name inside double quotes.
2016-12-21 21:40:50 +01:00
Hallvard Furuseth
72c893fc82 Mention MDB_PREV_MULTIPLE along with MDB_NEXT_MULTIPLE 2016-12-15 22:12:45 +01:00
Hallvard Furuseth
e539654051 ITS#8504 Fix prev commit: mc_error, #ifdef SIGPIPE
Never clear mc_error, we could lose a failure in the other thread.
2016-12-13 18:50:45 +01:00
Lorenz Bauer
3e7a8e26e6 ITS#8504 mdb_env_copyfd2(): Don't abort on SIGPIPE
Return EPIPE instead.
2016-12-13 18:47:13 +01:00
Hallvard Furuseth
65e95ffccf ITS#7377 Catch mdb_cursor_sibling() error 2016-12-10 22:00:31 +01:00
Hallvard Furuseth
d78c80d902 Clean up and comment C_UNTRACK
Don't use it as a "cursor is tracked" hint in mdb_pages_xkeep().
It's been harmless so far, but would break after mdb_cursor_copy().
Checking m0 directly short-circuits better anyway.
2016-12-10 21:42:39 +01:00
Hallvard Furuseth
1fb0822b40 ITS#8355 cleanup
Drop ~C_EOF, pointless after 8c215aa970
2016-12-10 21:11:12 +01:00
Hallvard Furuseth
c0ff9a267a MDB_VL32 cleanup: MDB_env.mm_mapsize type 2016-12-10 17:17:28 +01:00
Hallvard Furuseth
e1be73c771 ITS#8542 mdb_dbi_open(): Protect mainDB cursors 2016-12-10 09:16:17 +01:00
Hallvard Furuseth
ffd13db3f8 Cleanup: Add flag DB_DUPDATA, drop DB_DIRTY hack 2016-12-09 00:03:36 +01:00
Hallvard Furuseth
ef066598b5 MDB_CP_COMPACT comments 2016-12-07 19:06:11 +01:00
Hallvard Furuseth
3dda2bfa4a doxygen cleanup 2016-12-07 19:04:19 +01:00
Hallvard Furuseth
72f875b48a Note functions which must set MDB_TXN_ERROR on failure
Other functions depend on them to do so.
For mdb_node_read(), instead remove such a dependence.
2016-12-07 18:55:21 +01:00
Hallvard Furuseth
08e4684d37 MDB_VL32 mdb_page_get(): Set MDB_TXN_ERROR on failure. 2016-12-02 07:00:08 +01:00
Hallvard Furuseth
a70200f15f Fix mdb_env_open() with MDB_VL32
Init me_rpmutex independently of MDB_NOLOCK.
Plug leaks on mdb_env_open() failure.
Tweak mdb_env_close0() to handle the rearranged mdb_env_open().
2016-12-02 05:22:11 +01:00
Hallvard Furuseth
f3ab0d23a4 Factor out refreshing sub-page pointers 2016-12-01 21:17:42 +01:00
Howard Chu
fa83b25ec5 More for ITS#8406
xcursor fixup depends on init state
2016-11-29 19:19:45 +00:00
Howard Chu
e58db7d556 More for ITS#8406
Revert excess cursor fixup
2016-11-12 23:11:20 +00:00
Hallvard Furuseth
f7e85d7804 Only set me_mfd if needed. Drop unused read access. 2016-10-04 21:16:27 +02:00
Hallvard Furuseth
77845345ca ITS#8505 Clarify fork() caveat, mdb_env_get_fd(), flock->fcntl. 2016-10-04 21:16:27 +02:00
Hallvard Furuseth
6355dade31 ITS#8505 Protect parent from fork()-pthread_exit() 2016-10-04 21:16:27 +02:00
Hallvard Furuseth
04acac634a ITS#8505 Set FD_CLOEXEC for me_mfd,env_copy as well 2016-10-04 19:12:58 +02:00
Hallvard Furuseth
15666878af Move opening files to mdb_fopen()
No change in functionality.
2016-10-04 19:12:58 +02:00
Hallvard Furuseth
cdc3f9cc2d Factor filename handling out to mdb_fname_*()
No change in functionality, except needs less mallocing.
2016-09-25 15:40:08 +02:00
Hallvard Furuseth
67fb3c746a ITS#7992 Tighter utf8_to_utf16(), fix errcodes
The 0xFFFD check seems due to misleading MultiByteToWideChar() doc.
Bad UTF-8 gives 0xFFFD in the output string, not the return value.
2016-09-25 15:40:08 +02:00
Hallvard Furuseth
d87ee20e0b Clean up strange fcntl result check
...and check !MDB_CLOEXEC in an 'if' rather than '#if'
to match its non-zero usage.
2016-09-25 08:12:56 +02:00
Hallvard Furuseth
3dd2d207d7 Drop spurious Errcode() call 2016-09-25 08:12:56 +02:00
Hallvard Furuseth
cdcf9da600 ITS#7682 F_NOCACHE: Allow error, skip any O_DIRECT
We can run without F_NOCACHE if it fails. And we do not know
what combining it with O_DIRECT means, if a system has both.
2016-09-25 08:12:56 +02:00
Hallvard Furuseth
26e226b2a8 Silence warning for initializer "mdb_copy my = {0}"
1st struct member was not a scalar.
2016-09-03 09:41:30 +02:00
Howard Chu
45a88275d2 ITS#8489 reset cursor EOF flag in cursor_set
It usually gets done anyway, but one of the fastpath shortcuts
bypassed this step.
2016-09-01 00:41:35 +01:00
Hallvard Furuseth
da4443a9b3 Clean up MDB_USE_ROBUST #defines
Make explicit and default nonzero equivalent. Parenthesize.
2016-08-21 23:22:18 +02:00
Hallvard Furuseth
36d374ba9e Fix mdb_page_list() message 2016-08-21 23:20:22 +02:00
Howard Chu
c6510147ce ITS#8481 make shared lib suffix overridable 2016-08-19 17:24:25 +01:00
Howard Chu
7e2290c52b MDB_VL32 plug rpage leak
mdb_cursor_set wasn't unref'ing as intended.
2016-08-11 21:30:56 +01:00
Hallvard Furuseth
77d522d151 Comment MDB_page 2016-08-02 22:17:54 +02:00
Hallvard Furuseth
8670805b4f Doc fixes: VALID_FLAGS, mm_last_pg, mt_loose_count 2016-08-02 22:16:09 +02:00
Hallvard Furuseth
58b0ce50b3 mdb_size comments 2016-08-02 21:04:17 +02:00
Hallvard Furuseth
4d47e89f4d MDB_VL32: Switch to mdb_size_t formats PRIu64 & co
Drop macro Y=MDB_FMT_Y, add Yu/Yd=MDB_PRIy(). Replace
Y"d..." -> Yd"...", Y"u..." -> Yu"..." / MDB_SCNy(u)"...".
2016-08-02 21:02:35 +02:00
Hallvard Furuseth
65d9791ada Refactor mdb_page_get() 2016-07-23 12:45:46 +02:00
Hallvard Furuseth
a43fcad8c6 MDB_VL32: Match mdb_size_t type with format modifier.
When using format modifier "ll" or "I64", use the matching
type unsigned <long long / __int64> rather than uint64_t.
2016-07-23 12:43:16 +02:00
Hallvard Furuseth
f25d716513 Fix MDB_INTEGERKEY doc of integer types 2016-07-23 12:18:01 +02:00
Hallvard Furuseth
0842f998ee Use mdb_size_t for line numbers in mdb_load
This matches the mdb_size_t entry counts.
2016-07-23 12:17:04 +02:00
Hallvard Furuseth
12ad38d67f Fix size_t/formats -> mdb_size_t for MDB_VL32 2016-07-23 12:16:20 +02:00
Hallvard Furuseth
dff8bafb36 Factor some MDB_VL32-related '#if's out to macros
Add MC_OVPG() + MC_SET_OVPG(), NEED_CMP_CLONG(), MDB_CURSOR_UNREF().
2016-07-23 12:11:34 +02:00
Hallvard Furuseth
32764bcb52 Factor out MDB_SIZE_MAX, MDB_FMT_Y, MDB_FMT_Z 2016-07-23 12:08:12 +02:00
Hallvard Furuseth
84610e65da Add error MDB_PROBLEM, replace some MDB_CORRUPTED
When problem is most likely in txn, not on disk.
2016-07-14 05:53:21 +02:00
Hallvard Furuseth
291c69ddbd ITS#8209 Tweak previous fixes
Some _aligned_malloc() doc seems to think arg NULL = user error.
Don't know if posix_memalign() pointer is defined after failure.
2016-06-29 06:25:37 +02:00
Hallvard Furuseth
5ea12b0be8 ITS#8209 MDB_CP_COMPACT: Handle empty or broken DB
Preserve DB flags (use metapage#1) when main DB is empty.
Fail if metapage root != actual root in output file.
2016-06-25 07:57:04 +02:00
Hallvard Furuseth
eb7bbed967 ITS#8209 MDB_CP_COMPACT fixes
Handle errors.  Fix cond_wait condition so mc_new
is the sole control var.  Drop specious cond_waits.
Do not look at 'mo' while copythr writes it.
2016-06-25 07:55:34 +02:00
Hallvard Furuseth
c4c7833d24 mdb_env_setup_locks: Plug mutexattr leak on error 2016-06-05 23:42:44 +02:00
Hallvard Furuseth
53a0fdf1be Init "locked" flag for SysV semaphores 2016-06-03 06:11:54 +02:00
Howard Chu
c367c1f696 ITS#8339 Solaris 10/11 robust mutex fixes
Check for PTHREAD_MUTEX_ROBUST_NP definition (this doesn't work
on Linux/glibc because they used an enum). Zero out mutex before
initing.
2016-06-02 21:01:27 +01:00
Howard Chu
b045bce260 ITS#8424 init cursor in mdb_env_cwalk 2016-05-15 00:44:54 +01:00
Hallvard Furuseth
e2b8b6448d Comment ovpage code in mdb_cursor_put() 2016-05-10 07:11:44 +02:00
Howard Chu
a04aad31c2 ITS#8412 fix NEXT_DUP after cursor_del 2016-04-26 12:52:21 +01:00
Howard Chu
37081325f7 ITS#8406 fix xcursors after cursor_del
Don't leave them uninit'd if they now point at a valid DUP node
2016-04-18 18:07:56 +01:00
Howard Chu
c8dbd772f7 mdb_drop optimization
If we know there are no sub-DBs and no overflow pages, skip leaf scan.
2016-04-09 20:42:45 +01:00
Howard Chu
8fff90db7e ITS#8393 fix MDB_GET_BOTH on non-dup record 2016-03-28 01:36:26 +01:00
Howard Chu
e46d78b7b0 MDB_VL32 - increase max write txn size 2016-03-02 16:29:54 +00:00
Howard Chu
3f62b727cc Tweak MDB_PREV_MULTIPLE for uninit'd cursor 2016-02-16 23:38:43 +00:00
Howard Chu
d909ab2f36 Tweak Win32 errmsg buffer 2016-02-15 00:07:04 +00:00
Howard Chu
5ef1908224 ITS#8324 Map NTAPI result codes to WIN32 codes 2016-02-14 23:53:05 +00:00
Hallvard Furuseth
fcac8d0774 ITS#7992 Fix memleak in prev change 2016-02-04 03:23:13 +01:00
Howard Chu
5f5f4dab9c Happy New Year 2016-01-30 12:54:32 +00:00
Hallvard Furuseth
e394e023e4 Fix MDB_VL32 mdb_cursor_count()/entrycount types 2016-01-28 19:45:01 +01:00
Howard Chu
3f62ddc81c MDB_VL32 change overflow page scan
Just check the requested page, don't worry about any other pages
2016-01-28 14:23:02 +00:00
Howard Chu
5bf313e820 ITS#8363 Fix off-by-one in mdb_midl_shrink 2016-01-28 04:21:32 +00:00
Howard Chu
6f653ca205 MDB_VL32 more for 1ba5adb2ec
fix 32bit overflow in set_mapsize
2016-01-27 11:48:22 +00:00
Howard Chu
8f88b1b0ba ITS#8324 fix a6ccef73ed
Removing the WRITEMAP test dropped this into the MDB_VL32 code path,
which was wrong.
2016-01-26 22:13:01 +00:00
Howard Chu
8c215aa970 ITS#8355 fix subcursors
make sure C_DEL gets reset in subcursor after it moves.
2016-01-24 11:35:42 +00:00
Howard Chu
b5018e26bc Update WRITEMAP doc 2016-01-23 16:51:42 +00:00
Howard Chu
20dec1f69b WIN64 needs off_t redefined too 2016-01-20 01:30:57 +00:00
Howard Chu
a6ccef73ed ITS#8324 fix c4e31434c7
Actually, there is no guarantee that Windows will map newly written
data, so we need VirtualAlloc even for non-WRITEMAP.
2016-01-16 17:13:25 +00:00
Hallvard Furuseth
d6995599b4 lmdb.h Caveats: Reserved vs. actual mem/disk usage 2016-01-12 23:18:06 +01:00
Howard Chu
15880014ba ITS#8346 fix typo in prev commit 2016-01-12 07:13:48 +00:00
Howard Chu
a5bf1648c6 ITS#8347 fix off-by-one in VL32 purge 2016-01-11 20:25:33 +00:00
Howard Chu
447683a824 ITS#8346 free last txn0->mt_rpages in env_close 2016-01-11 20:17:42 +00:00
Howard Chu
27b1c5f366 Add MDB_PREV_MULTIPLE
Logical counterpart to GET_MULTIPLE, NEXT_MULTIPLE
2016-01-07 18:28:29 +00:00
Howard Chu
5db0b54ca1 Fixups for env_copy with large files
wsize was being truncated to 32bits on Windows.
Only try to write 1GB at a time on Windows64;
larger writes fail with ERROR_WORKING_SET_QUOTA.
2016-01-02 12:31:49 +00:00
Howard Chu
53f624bff0 MDB_VL32 fix cursor_unref - ignore cursor with empty stack 2015-12-28 20:31:09 +00:00
Howard Chu
1c2a5888cf ITS#8342 MDB_VL32/WIN32 - close file mapping handle in env_close 2015-12-28 19:38:32 +00:00
Howard Chu
c3852f29af MDB_VL32 tweak prev commit 2015-12-28 19:38:32 +00:00
Howard Chu
7b9928ced4 MDB_VL32 - prevent mixing with non-VL32 builds 2015-12-23 23:02:06 +00:00
Howard Chu
90d1ee279f MDB_VL32 - Fix me_rpmutex usage 2015-12-22 20:24:26 +00:00
Howard Chu
825ab2ad52 MDB_RESERVE doc
Add mdb_put text to mdb_cursor_put description for people who
fail to draw logical conclusions.
2015-12-22 18:30:41 +00:00
Howard Chu
9d75a82ae1 MDB_VL32 - fix prev commit
Only tweak length for read-only envs
2015-12-22 06:13:25 +00:00
Howard Chu
5eb25c5cb9 MDB_VL32 - fix for Win32 read-only envs
We can't map with MEM_RESERVE because that requires write access
to the underlying file/section. Mapping with the default (MEM_COMMIT)
requires that we don't map past the end of the file.
2015-12-22 05:20:33 +00:00
Howard Chu
3a2ac24f5b MDB_VL32 - track overflow pages too
Otherwise they'll fill up the txn's pageref list when traversing
a DB with lots of overflow records.
2015-12-22 04:22:01 +00:00
Hallvard Furuseth
6534b804a5 mdb_page_alloc(): Use original snapshot of freeDB.
I can't help thinking this should be safer, and useful somewhere.
2015-12-22 04:21:23 +00:00
Hallvard Furuseth
33025182cc Prep for next commit: C_WRITEMAP, C_ORIG_RDONLY.
No change in behavior.
2015-12-22 04:21:13 +00:00
Hallvard Furuseth
397d85d13c Pass cursor to mdb_page_get(), mdb_node_read().
No change in behavior.
2015-12-22 04:20:41 +00:00
Howard Chu
209b56fead ITS#8324 fix for WRITEMAP
We called FlushViewOfFile with (map,mapsize) which worked fine
when we had allocated the entire map already. Now we have to make
sure to only flush as much as was actually written. Add a numpgs
argument to tell how much to flush in env_sync0().
2015-12-21 02:36:20 +00:00