Siaqodb 3.6 released

We just released a new version which brings the most requested feature in last period: OID is not mandatory anymore in a persistent Type.
So instances of following class may be stored by Siaqodb:

public class Person
    {
        public DateTime BirthDate { get; set; }
        public string Name { get; set; }
        public string FirstName { get; set; }
        private int age;
    }

There are required more details how we implement it and how the engine will detect what objects need to be updated and what are seen as new objects since OID may miss from class definition.
Important to say first is that the new version is fully backward compatible, also important is that if you add OID property to
a persistent type, it will be still managed by the engine, so autoincremented; this can be still very useful in many scenarios and by having OID along with object you can update them directly or compare with existing objects in DB.
However OID still exists internally, the engine still identify the uniqueness of the object by OID.

For the types that not have defined OID property,when objects are loaded from database we keep them in a “weak cache structure”, so when a object it is stored, the engine check first if its reference exists in cache, get the OID if exists and then update the object, if the reference is not found in the cache then the engine will consider object as new and will insert it in the database. By a “weak cache structure” we mean that we cache only WeakReferences of objects, so if an object is not referenced anymore by your app, it can be collected by the garbage collector.
Also the cache automatically clean-up empty weak references, you don’t have to worry or configure anything about it.

Another thing is that if later on, in project life-cycle,you would need OID along with object, you can add the property at any time to the class definition and will be filled by the engine automatically with no extra overhead.

Enjoy!

Siaqodb 3.5 released

We just released Siaqodb 3.5 which includes Siaqodb for WinRT and also Siaqodb for WindowsPhone 8.

The most exciting is the release for WinRT, see our previous post for more info. In the current version we added indexes and all other features of version 3.5, so Siaqodb has now a fully async API. We are also happy that the approach we took with async API and “async LINQ” queries is the same with the approach Microsoft took with EntityFramework 6.0, so .ToListAsync(), .CountAsync(),.FirstAsync() etc.

Also important release is the Siaqodb for WindowsPhone 8 release, the SDK for WindowsPhone 8 has some important changes, for example one that affected Siaqodb and did not compiled was the fact that now TypeInfo is a type in System.Reflection and we had same type defined inside Siaqodb so it was a namespace conflict, luckily it was not public, so we could refactor easy. Having Windows 8 kernel, WindowsPhone 8 SDK brings all async stuff, so it opens door to bring our async version also to WindowsPhone 8, stay tuned.

As new features in this version most important is the new Index storage, now all indexes resides in a BTree structured files and not in a AVLTree anymore. With this change performance is much better in most of the cases and we also expect much more stability.

We bring also a new utility class: SiaqodbUtil through you can now Shrink your database files and also Repair persisted object if they got corrupted, see the documentation how to use it.

Download it now and enjoy new features!

Siaqodb – local database for Metro style apps

We are very proud to announce that we finally ported Siaqodb to .NET profile for Metro style apps (NETFX_CORE) and fully async API. We have ported previously on several platforms but now it took longer because about 80% of the methods had to be rewritten. Until now we have used in our code preprocessor directives like #IF SILVERLIGHT, #IF MonoTouch etc, because it was easier to maintain in this way Siaqodb sources for multiple platform, but NETFX_CORE could not be used in a similar way because the sources are changed too much as result of a few things, most important:
1.”async methods”
2. Reflection changes

We know portable class library but shortly: it doesn’t work for Siaqodb.

To pass validation for Windows 8 App Store, your Metro style app should call all the I/O operations async, which means a lot for the Siaqodb library. The methods that may fall into calling a I/O operation must be async and they have to be called with “await” in case you need the result returned for further processing; and await can be used only in a async method, so all methods from the lowest layer to the uppermost layer must be async.

Another important aspect for Metro style apps:
“A Metro style app runs in a security container with limited access to the file system, network resources, and hardware. Whenever a user installs an app from the Windows Store, Windows looks at the metadata in the Package.appxmanifest file to figure out what capabilities the app needs to function. For example, an app might need to access data from the Internet, documents from the user’s Document Library, or the user’s webcam and microphone. When the app is installed, it displays to the user the capabilities it needs, and the user must grant permission for it to access those resources. If the app doesn’t request and receive access to a resource it needs, it will not be allowed access to that resource when the user runs it.”
(source MSDN)

Just as you can access in Silverlight the IsolatedStorage, in a similar way you will have full and unrestricted access to app’s local folder found in C:\Users\\AppData\Local\Packages\. Here you can create, modify and delete files and folders without asking user’s permission. Local data access is provided using Windows.Storage.ApplicationData.
So, for opening a Siaqodb database you have to pass as database name the folder where your data will be stored like:


      Siaqodb siaqodb = new Siaqodb();
      await siaqodb.Open(ApplicationData.Current.LocalFolder);

As you know the only query engine for Siaqodb is LINQ and Siaqodb LINQ provider is implemented by extending IEnumerable interface; bad news is that currently there is not shipped any IAsyncEnumerable interface that would support LINQ and also having a method signature like
public async Task> GetEnumerator()
or/and IAsyncEnumerator where MoveNext() method is async, and -very important- Microsoft says that such a interface/s will not be shipped for now.

So we had to find a way to use LINQ but also to load objects from database async as result of that LINQ statement. We’ve made our IAsyncEnumerator which is used by most methods within our LINQ provider. The ISqoQuery interface still implements IEnumerable but when it’s called its GetEnumerator() method, an exception is thrown and client API must use our X_Async() methods instead. Basically we parse lambda expression in the same way as we did before, but when objects need to get materialized, we should call an async method since it may fall in a I/O operation.

For the standard .NET version, Siaqodb LINQ provider can optimize following LINQ operations by re-defining new methods for:

Any(...)
Count(...)
First(...)
FirstOrDefault(...)
Join(...)
Last(...)
LastOrDefault(...)
OrderBy(...)
OrderByDescending(...)
Select(...)
Single(...)
SingleOrDefault(...)
Take(...)
ThenBy(...)
ThenByDescending(...)
Skip(...)
Where(...)

Most of the methods above will throw exceptions now, but the following methods must be used for Siaqodb for Metro Style apps:

AnyAsync(...)
CountAsync(...)
FirstAsync(...)
FirstOrDefaultAsync(...)
LastAsync(...)
LastOrDefaultAsync(...)
OrderBy(...)
OrderByDescending(...)
Select(...)
SingleAsync(...)
SingleOrDefaultAsync(...)
TakeAsync(...)
ThenBy(...)
ThenByDescending(...)
SkipAsync(...)
Where(...)

And to get result of a LINQ query that returns ISqoQuery you always have to call ToListAsync() method, otherwise, fetching the IEnumerable will throw exception.

So let’s see some code:

Siaqodb siaqodb = new Siaqodb();
await siaqodb.Open(ApplicationData.Current.LocalFolder);
//store the first person
await siaqodb.StoreObject(new Person() { FirstName = "Michael", LastName = "Smith" };);

//load all persons stored in database
var allPersons=await siaqodb.LoadAll<Person>();

//make a LINQ query
var persons=await(from Person pers in siaqodb
                        where pers.FirstName=="Michael"
                        select pers).ToListAsync();

The current version is in BETA stage, you can download it and use it as trial for 90 days
The Indexes are the only missing feature in this version( but still read performance is very good), all the rest of features of Siaqodb 3.2.0.5 are included in the Siaqodb for Metro style apps.

Very important, we ported also all UnitTests and all passed!

Since this is a BETA version, we wait for your feedback, so if you have questions/remarks shout us at support at siaqodb dot com.

So go to our download page and grab it now, you have a lot of examples that can be found in MetroExample project included in the setup!

SiaqodbManager now on Mac OSX

Because we saw high interest for Siaqodb from Unity3D developers, especially this being reflected in our sales for this platform, we ported
SiaqodbManager now on Mac OSX but also on any OS that support Mono, you can start it from terminal by: mono /User/…/SiaqodbManagerMono.exe

SiaqodbManagerMono.exe is WindowsForms based app so it does not look native Mac OSX app, but it is working fine and can be very helpful for developers, you can run ad-hoc LINQ queries and it has all features SiaqodbManager has on Windows. Only one limitation exists: on LINQ editor, the LINQ statements are not highlighted like on Windows version.
Enjoy!
siaqodbManager on Mac OSX

Siaqodb 3.2 released

We just released version 3.2 which brings great new features:

  • support for Dictionarytypes
  • add support for unlimited size jagged arrays types
  • add support unlimited nested Lists ( ex: List<List<int>>)
  • add support for IList items properties in LINQ Include() method (ex: .Include(“Employees.Nationality”); where Employees is IList)
  • performance improved on insert with 30% in same cases
  • new methods for fast insert: siaqodb.StartBulkInsert(…) and siaqodb.EndBulkInsert(…)
  • better information about exceptions
  • other improvements

So now objects of following class may be stored in Siaqodb database:

public class MyClass
{
public int OID{get;set;}

public List<List<int>> JaggedList {get;set;}

public int[][][] Matrix;

public Dictionary<int,AnotherClass> Dictionary{get;set;}
}

In LINQ queries over new supported types Siaqodb may optimize following methods:
-ContainsKey(…) of a Dictionary
-ContainsValue(…) of a Dictionary
but also it can optimize and compare a list with sub-list by its elements not by reference.
Examples:

var query=from MyClass m in siaqodb
where m.Dictionary.ContainsKey(10)
select m;

//or:
AnotherClass ac=new AnotherClass();
ac.Id=10;
var query=from MyClass m in siaqodb
where m.Dictionary.ContainsValue(ac)
select m;

Both runs optimized and at second example it is compared values of properties/fields of ‘ac’ object instead of comparing by its reference so you are able to pull from database data you want.

Next example:

List list=new List();
list.Add(1);
list.Add(2);

var query=from MyClass m in siaqodb
where m.JaggedList.Contains(list)
select m;

In this case it runs optimized and again it is compared each element of inner list from database with elements of ‘list’ object provided to query.

Other new methods that can be used when a lot of data need to be stored, are StartBulkInsert(…) and StartBulkInsert(…), example:

siaqodb.StartBulkInsert(typeof(Order));
try
{
for (int i = 0; i < n; i++)
{
....
Order order=new Order();
s_db.StoreObject(order);
}

}
finally
{
s_db.EndBulkInsert(typeof(Order));
}

We hope you’ll enjoy new features,if so, go to download page and grab the latest version.

Siaqodb 3.1 released

We just released version 3.1 of Siaqodb, as new features we added:

  • new method: StoreObjectPartially(…) added to Siaqodb class. Through this method you can save only a certain properties of an object and not a full object. This method arise as a feature request after partial object loading and eager loading feature was added, so it is possible that you did not load the object fully from database(complex nested objects was not loaded) and when this object would be saved, all complex properties values will be set to null. To avoid that and also in other scenarios you can use now new method :
    StoreObjectPartially(object obj,params string[] properties);
    
  • optimize LINQ methods:OrderBy(…), OrderByDescending(…), ThenBy(…), ThenByDescending(…). All these methods are optimized now by our LINQ provider implementation.
  • add support for storing directly generics types. Now the following type is storable:
    
    class MyClass<T>
    {
        public List<T> myList;
        public int OID{get;set;}
    }
    
  • added Replace parameter on LoadingObjectEventArgs class. There exists cases when you want to implement a caching system and, when object that has nested objects is loaded from database, instead of letting Siaqodb engine to load that object from database you can check by OID and Type if object exists in your cache and if yes, assign it to Replace parameter, so the engine assign your provided object and return to the caller.
  • new method EnableOptmisticConcurrency(…) added to SiaqodbConfigurator class useful when you work with SiaqodbOffline (in Sync Framework Provider context).
  • Enjoy new version and a Happy New Year from Siaqodb team!

    Siaqodb v3.0 released

    We happy to announce that we just released version 3.0 of Siaqodb.
    There are not many extra things since our BETA release so for details of features see previous blog post.

    We also released SiaqodbManager 3.0 (which was not in our BETA) that fully support database created with version 3.0, the nested objects or Lists/Arrays of nested objects may be opened by clicking on the cell, see screenshot bellow:
    SiaqodbManager3.0

    Finally, we released also Siaqodb Sync Framework Provider which is based on Microsoft Sync Framework Toolkit .

    Siaqodb Sync Framework Provider is available for all platforms: .NET, Silverlight,WindowsPhone7, MonoAndroid, MonoTouch, CompactFramework.
    There are many cool things to be said about this release so we will dedicate separate posts about it in near future.

    Stay tuned and enjoy the new version!

    Siaqodb and nested objects

    We are excited to announce Siaqodb 3.0 Beta version ready for download. It is a major version and will be available for all platforms: .NET, Silverlight, WindowsPhone7, Mono, MonoTouch, MonoAndroid and Unity3D. The main feature we introduce in this version (the most requested by developers) is support for nested objects and Lists/Arrays of complex objects, Siaqodb being able now to store full graph of object no matter how complex it is.

    Example of  a storable object/s :

    public class Order
        {
            public int OID { get; set; }
            public Customer OrderCustomer { get; set; }
            public List<OrderDetail> Details{get;set;}
            public DateTime OrderDate { get; set; }
            public int OrderNumber { get; set; }
        }
        public class OrderDetail
        {
            public int OID { get; set; }
            public Product DetailProduct { get; set; }
            public decimal Price { get; set; }
            public decimal Quantity { get; set; }
        }
        public class Product
        {
            public int OID { get; set; }
            public string Name { get; set; }
        }
        public class Customer
        {
            public int OID { get; set; }
            public string Name { get; set; }
            public Address Address { get; set; }
    
        }
        public class Address
        {
            public int OID { get; set; }
            public string Street { get; set; }
            public string City { get; set; }
        }
    

    And now query the db:

     var query = from Order o in siaqodb
                         where o.OrderCustomer.Name == "CustomerX"
                         select o;
    

    Above query runs optimized, only objects that match with criteria will be loaded/created from database.
    Also very important, in queries you can compare complex objects like:

                Customer customer = new Customer() { Name = "CustomerX", Address = new Address() { City = "Berlin" } };
    
                var q = from Order o in siaqodb
                        where o.OrderCustomer == customer
                        select o;
    

    The LINQ provider will not compare by reference the objects stored in db, it will make a deep comparison of every property/field and if objects match, instance will be created.

    As you know from one of our first blog posts , we did not implemented this feature for some reasons, the most important one was performance and we suggested to use JOINs instead of nested objects. But ORMs like EntityFramework implemented features like ‘Lazy Loading’ and ‘Eager Loading’, so developer may decide what nested objects will be loaded and when, in this case developer can avoid performance problems depends of app business needs. Well… objects databases vs. ORMs+relational databases is out of scope of this post, but short notice here: Siaqodb store complex objects by keeping pointer/s of childs objects along with object; with this pointer the engine is able to seek directly in database file without any scan or index fetch, so time to look out for nested objects is ZERO, comparing with ORMs+relational dbs it is big difference as performance.
    So what if, for example, you want to load orders but you don’t need all nested objects +lists (Customer,OrderLines etc), in this case we let developer configure how objects are loaded so if you set to a certain Type to not load related objects, then all properties(complex types) will not be loaded (will be NULL) by default, but still you may .Include(…) properties like:

                SiaqodbConfigurator.LoadRelatedObjects<Order>(false);
                SiaqodbConfigurator.LoadRelatedObjects<Customer>(false);
    
     var query1 = siaqodb.Query<Order>().Where(o => o.OrderNumber > 2).Include("OrderCustomer")
                                                                      .Include("OrderCustomer.Address");
    //or similar:
      var query2 = (from Order o in siaqodb
                             where o.OrderNumber > 2
                              select o).Include("OrderCustomer").Include("OrderCustomer.Address");
    

    In above queries will be loaded complex properties OrderCustomer and also Address of Customer; Details property will not be loaded.
    Look in Examples projects ..\NestedObjectsExample.cs for the complete example.

    We hope you enjoy the new features and since this version is in Beta stage we wait for bugs, remarks, suggestions at support at siaqodb com

    Siaqodb on Unity3D

    We are glad to announce that we successfully port Siaqodb to Unity3D game platform so now our database engine can be used as local database for Unity3D games on PC, Mac, iOS and Android.

    Features of Siaqodb for Unity3D:

    • it is fully managed, written completely in C#
    • only one assembly, no 3th party dependencies
    • only two Mono dependencies: System.dll and System.Core.dll
    • small footprint(siaqodb.dll has 240KB)
    • it has very very simple API, LINQ is only query engine (not need to know SQL or other query language)
    • persist your game data with one line of code:
    localDB.StoreObject(new Player(){Name="Player1",Score=100});
    
    • and retrieve back using LINQ:
    • var query = from Player p in localDB
                       where p.Name == "playerName" && p.Score>100
                       select p;
      
    • your data is stored/retrieved in same way on Windows, Mac, iOS and Android, no code change for specific platform
    • zero config
    • not need Unity3D Pro edition, it works with all Unity3D versions and platforms
    • and many more…

    Download Siaqodb for Unity3D now and start using it in your games!
    Inside package you will find also a complete demo how Siaqodb may be used for all type of games: PC, Mac, iOS and Android.

    And yeah, since we just released we want to offer 20% discount for first 10 purchases, just put “siaqodbUnity3D” as coupon code and discount will be applied, so hurry up!

    Siaqodb support now IList and Arrays

    Today we released Siaqodb v.2.6, as release notes:

  • Indexes are now persisted in db as AVL tree structured files
  • Support for variable length of String by using TextAttribute
  • Support for blobs (array of byte in efficient way)
  • Support for Array of supported types
  • Support for IList and IList<T>
  • new API method: LoadAllLazy<T>() which load all objects in lazy mode, only when object is accessed by index or by enumerator, it is loaded from db and instance is created.

    Indexes in previous versions was built in background in memory and kept in AVL tree but for very large databases that was still a problem because app may need indexes after db is opened and if they are not yet ready, was not used. Now this problem is solved, indexes are persisted in db as AVL structured file accessing by queries being still as fast as before.

    Variable length of strings: until now Siaqodb stores by default string as max 100 chars, it was also possible to increase that by using MaxLength attribute, but still it was a problem if your strings properties has not appropriate length, one being very big one being small and it was waste of disk space. Now by using Text attribute, string is stored as is, at length it has without any extra byte.

    Support for IList and Array for object members, this was a very demanded feature and staring with this version it is fully supported,also queries over this member are optimized ex:

    public class Post
    {
    public int OID{get;set;}
    public List Tags;
    public string Name;
    }
    ..........................
    
    var query=from Post p in siaqodb
    where p.Tags.Contains("MVVM")
    select p;
    

    When above query runs, engine create only instances of Post type which satisfy condition.

    SiaqodbManager support view/edit/add Arrays/Lists too:
    EditArrays

    Download latest version from download page.
    Happy coding ;)

  •