On 3. mars. 2008, at 09.44, Paul J Stevens wrote:
Jan-Henrik Haukeland wrote:
On 2. mars. 2008, at 22.09, Paul J Stevens wrote:
I've been thinking about this. This implies that you can't use multiple statements within a single transaction (at least with sqlite). Is that correct? Because that would make code like below invalid, right?
Not necessarily, though a fair chance that it will not work. The indicator is this SQLite error message "SQL statements in progress".
I am aware of that caveat in the docs. Is that a limitation of sqlite per se? Or rather how libzdb talks to sqlite?
I believe it is a combination. Using PreparedStatement_executeQuery() the prepared statement is reused in the ResultSet and not closed. One place where this is a problem is in an explicit transaction. Upon a commit or rollback, at least SQLite check for open statements and issue a "SQL statements in progress" error, unless Connection_clear() is called to explicit close the statement.
For Connection_execute() and Connection_executeQuery() which also use a statement internally, the statement is closed upon the next call to any of these methods and therefor does not have this problem. But for PreparedStatement_executeQuery() the same internal statement is reused, and since a prepared statement is longer lived [1] than a ResultSet it may be a problem. A better option than to reuse, is probably to create a new statement for the ResultSet when PreparedStatement_executeQuery() is called, though this need more research which I haven't had time to prioritize. Until then, a workaround is available by calling Connection_clear() at appropriate places to close prepared statements explicit.
The reason I'm asking is because I'm still trying to get a feel for libzdb's behaviour and implicit expectations. A few things I've noticed:
- I can't use Connection_PreparedStatement on a postgresql
connection until I've issued a Connection_executeQuery. Doing so segfaults in libzdb. I've reported this one already, and I can easily work around this.
Its on my TODO list and I hope to have it moved over to Martin's TODO list :)
- I'm getting stack corruption and segfaults in certain hotspots in
my code that use glib's GObject interface throught the GMime code. Now is GObject a known not-threadsafe interface, and putting mutexes around my calls solves the segfaults. But thing is: my code is (still) single-threaded (apart from the reaper thread). So apparently thread semantics apply even to single- threaded apps when using libzdb. If that is true, I need to start studying on threads pronto because I havent a clue as to how thread(un)safe my code is at the moment.
Linking with pthread makes your application threaded, per def, *but* as long as you only use one thread, i.e. the application main-thread you should not need to consider yourself with tread-safeness nor synchronizing shared resources. This is also the case if you run libzdb with a reaper thread since it does not affect any of your application data structures.
Thread-safe is one thing, reentrant is another and since I know you are using libevent, you may also want to check your event handlers. Even a single threaded application may have reentrancy problems - think signal handlers, and using libevent may introduce reentrancy problems in your code which may look like a thread bug. Maybe linking with pthread/libzdb exposed these bugs, but I don't think its the root cause.
- Valgrind is generating massive amounts of 'invalid read' warnings
in the ResultSet_getXXX code. Whether this results in segfaults is dependent on the backend. The mysql driver appears to segfault where the postgresql driver simply returns null values. I havent yet tested sqlite. I'm left wondering if this too is thread related.
Well, this is unexpected. Not that valgrind issues lots of warnings, but that you get SIGSEGV from using ResultSet_getXXX. Sounds like a Heisenbug (http://en.wikipedia.org/wiki/Heisenbug) someplace. Hopefully not in libzdb. One thing to check is PreparedStatement_setXXX. String or blob values set in a PreparedStatement_setXXX method are set by *reference* and if those references has disappeared somehow before doing a PreparedStatement_execute.. there will be Heisenbugs.
However, since I only just finished the code changes that de- globalize the result and connection pointers (leaving only the pool pointer as a global) I havent had enough time yet to do some thorough backtracking.
keep you posted.
Appreciated!
[1] A prepared statement is closed when its parent Connection is returned to the pool. Though for efficiency it should live longer. libzdb should/may in the future internalize and cache them. I have not verified this, but I hope the database backends already does this.