Start a new topic

IOException: The process cannot access the file because another process has locked a portion of the file.

Hi

Just trialing VistaDB and started a multi threaded application that uses parallel but keep getting the following error:


IOException: The process cannot access the file because another process has locked a portion of the file.


I have setup the connection to use MultiProcessReadWrite


Is there something else that I am missing?


Full Error:

VistaDBConnectionStringBuilder connStringBuilder = new VistaDBConnectionStringBuilder();

                connStringBuilder.DataSource = $"{dbFileRoot}DDMain.vdb6";

                connStringBuilder.Password = dbPassword;

                connStringBuilder.OpenMode = VistaDB.VistaDBDatabaseOpenMode.MultiProcessReadWrite;


Okay, I wanted to eliminate some other possibilities, but the further information you gave indeed tends to eliminate them, so that does sound like a lock timeout most likely due to contention.


As an embedded database which supports distributed (multi-process) operation, the VistaDB engine uses locking via the OS and filesystem to negotiate access between separate connections--which might be in different processes.  Because there's no central server to coordinate them, it can't detect deadlocks, so it relies on timeouts for lock requests which take too long.


As a consequence, if there are too many connections vying for the same resource or holding it for a long operation or query they can start to get lock timeout errors.  These can be caught--and the query or operation retried--but before doing that there may be some other things you can do to reduce their likelihood and potentially eliminate them.


If you're opening a connection and disposing it after each query or operation (such as with a using statement)--which is a common pattern--you will probably want to enable connection pooling so the engine can recycle the connection objects.  Set Pooling to true in your connectionStringBuilder to turn it on.


Consider the degree of parallelism you are attempting to use (it could be too high) and what resources are being contended for.  Queries of the same table (and especially modifications to the same table) will be in contention with each other a lot and could starve each other out of locks on the table.  If you're breaking down write operations to the same table and trying to parallelize them, it could end up hurting overall throughput because each one will need to lock the same table and be largely serialized, anyway.  If the operations are already coming from separate processing and can't be naturally combined into fewer (still simple) operations, you might try tuning down the degree of parallelism (have fewer threads producing operations, particularly when going to the same table).


If your app will be running multi-threaded but all within a single process, you might try SingleProcessReadWrite mode.  This mode uses in-memory locking rather than using the filesystem and (as of 6.1 which you are using) should be more fair in awarding locks in the order the requests were made--whereas the filesystem has to keep trying periodically and gets granted more randomly and potentially unfairly.  You'll still want to consider the degree of parallelism, etc, because it's still using the locks in the same way, but it might handle a little more parallelism before getting lock timeouts.


If you only get timeout errors occasionally then a try/catch at an appropriate level can retry the operation or query (with some sane limiters).  See the ErrorId property and the Contains method on VistaDBException for testing for the specific error 163 (or VistaDB.Diagnostic.Errors.dda_LockTimeout):  VistaDBException Class Members.


    -Rob


PS. Oops, updated to correct the error--I got confused with the other recent thread.


Hi Rob, the inner exception stack trace:


   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)

   at System.IO.__Error.WinIOError()

   at System.IO.FileStream.Lock(Int64 position, Int64 length)

   at VistaDB.Engine.Core.IO.StorageHandle.LockFileStream(UInt64 pos, Int32 count)

   at VistaDB.Engine.Core.IO.StorageHandle.StreamLocksDictionary.StreamLocks.LockPosition(UInt64 position, Int32 count)

   at VistaDB.Engine.Core.IO.StorageHandle.Lock(UInt64 pos, Int32 count, UInt64 storageId)

   at VistaDB.Engine.Core.Indexing.Index.OnLowLevelLockStorage(UInt64 offset, Int32 bytes)

   at VistaDB.Engine.Core.DataStorage.LowLevelLockStorage(UInt64 offset, Int32 bytes)

   at VistaDB.Engine.Core.IO.LockManager.LockStorage()

   at VistaDB.Engine.Core.IO.LockManager.LockObject(Boolean userLock, UInt64 id, LockType type, Boolean& actualLock, Int32 lockTimeout)


The version is: 6.1.2.0


The connection is already open, this line that triggers this is a ExecuteReader:

VistaDBDataReader sr = command.ExecuteReader();


This is for a simple select command:

command.CommandText = string.Format(@"SELECT Id

                                          FROM AnalyEntity

                                          WHERE LogicalName='{0}' and OrgId={1};", e.LogicalName, e.OrgId);



This is running on windows


Please let me know if I can get any more information.


Are you getting a VistaDB error wrapping that error, or getting that exception directly?  The particulars of the VistaDB error(s) will probably shed more light on what situation is occurring.


Also, what version of VistaDB are you using, and on what OS platform (Windows, MacOS, Linux)?


And is this happening when trying to open a connection, or when executing a query?

Login to post a comment