Software Support > DOS/WinCab
Libraries
Chris:
I'm referring to them as "filters". The default concept is to be able to pop up a menu to select to filter on decade, genre or artist, but there's also the idea of "user definable" filters that would be equivalent to your Libraries.
There are actually already definitions in the code for BTN_FILTER_SELECT, BTN_FILTER_CLEAR, BTN_FILTER1 through BTN_FILTER10 (for direct access to a particular filter), so BTN_NEXTFILTER and BTN_PREVFILTER should be easy enough. BTN_FILTER_CLEAR would be equal to your BTN_ALLLIB.
It is not necessarily far off, as there is code in place for parts of itr already. Really part of the issue is that it wil require a lot of redesign of the structure of jukebox.ini and skins, as I need to design a way to do pop-up dialogs that are completely skinnable that don't interfere with the playing music, can be accessed either through direct keystrokes, joystick, mouse, touchscreen, etc., and can be used for functions other than just selecting a filter. It really is a major design change. Also, the way the database is designed, using filters on a large library is slow; I need to either redesign my database to support indexes or put in a proper database backend like SQL. In fact, I do have the beginninghs of an SQL backend in place based on SQLite but I'm having trouble getting it to compile in DOS.
Thre is actually all kinds of dormant code in DWJukebox for half-finished features: user-definable timers, macro support, switching skins on the fly, multi-line text rendering for long song names, etc. How much of it will ever see the light of day remains to be seen.
ex_directory:
Hi Chris,
Sounds like you are combining the ideas of a search and a library into your "filter" idea.
As a suggestion which you may already be doing, you could take this development in parts, the first being to have the concept of filters configured in the .ini, so no dialog box as search but again multiple BTN's to access them. When you talk about Genre's/Libraries then this will be a limited set for 'most' people so you get away with pre-configuring option, obviously the search really requires the pop-up dialog. That way you can deal with the skinning problem later.
With regards to the database issue, I guess you have a proprietary database solution in place at the moment, changing backend for one feature sounds like a lot of work. Is your database solution causing you problems in other areas? I once developed a proprietary database and later had to add indexes for sorting/filtering. I opted to use memory, loading the database in first, then using several passes of the full db to create new lists (pointers) in memory to the original master list.
Hey, don't talk to me about dormant code, I have been developing the same system (which is also live) since 1994 - my day job. Every now and then dormant (or supposedly dormant) code gets in the way. And then you are caught between we may need this one day so I'll need to spend time to keep it working / remove it to be asked for it six months later! :banghead:
May I say how impressed I am with wincab so far, from downloading to adding music to mod'ing an existing skin to match the colours of my cab was done in about 45 mins. Was used extensively on Saturday night and even after I give a quick demo I normally get questions, how do I scroll down again, how do I add a track, ..., not one question - cool :applaud:
Chris:
--- Quote from: ex_directory on March 31, 2008, 11:09:26 am ---As a suggestion which you may already be doing, you could take this development in parts, the first being to have the concept of filters configured in the .ini, so no dialog box as search but again multiple BTN's to access them. When you talk about Genre's/Libraries then this will be a limited set for 'most' people so you get away with pre-configuring option, obviously the search really requires the pop-up dialog. That way you can deal with the skinning problem later.
--- End quote ---
That is essentially exactly how it is planned to work. The dialog only appears if you press BTN_FILTER_SELECT, which would then list all the predefined filters or allow you to create a filter on the fly. This is a replacement for a full-on search, which would require an full on-screen keyboard.
--- Quote ---With regards to the database issue, I guess you have a proprietary database solution in place at the moment, changing backend for one feature sounds like a lot of work. Is your database solution causing you problems in other areas? I once developed a proprietary database and later had to add indexes for sorting/filtering. I opted to use memory, loading the database in first, then using several passes of the full db to create new lists (pointers) in memory to the original master list.
--- End quote ---
My "database solution" isn't even really a database. The DB file is loaded into memory verbatim as an array of strings. I have a bunch of helper routines written to access the data as if it were a database. This works because the data is for the most part read-only; the only data that changes is the play count and it only updates once per play, which is once every three to four minutes. Obviously this "solution" has serious scalability issues! In addition, the only real index is the song number. The path to the song is stored in a separate string array which is rebuilt on every run (that's what's happening when the library rescans on startup), unless the UpdateIndex option is set to False in which case it pre-loads the last path array and just trusts that all the songs are there and in the same place. (On an unrelated note, when the library is scanned, it is not looking for filenames; it looks at the last six digits of the file size and the first four characters of the filename and when it finds a match it assumes that is the same song. This allows a library built on DOS with 8.3 filenames to be compatible with a library built on Windows with long file names. That's why you don't see any file names if you open the database in a text editor.)
To support filters, essentially what I would do is at startup, check to see if either the database or filter definition has changed, and if so pre-scan the entire database to see what matches the filter. I would also do this for artists, genres and decades: create a separate index file for each possible genre, decade and artist (each one that exists at least once in the database) and pre-populate each with a list of integers representing the song ID's (position in the master string array) of each matching song. The ID3 specification allows for "artist" and "genre" to contain multiple values, so I have to account for that. None of this is spectacularly difficult, I just don't know if I want to manually build all the code for these indexes or just put in a proper database engine that will do all this for me and be more scaleable.
Ironically, the DB engine I'm looking at, SQLite, handles the database the same way I do: each record is a string and the entire database is held in RAM at once.
Either way I need to reorganize something, and once I start doing it I'm gonna break things. But all database access is virtualized through a couple of helper routines so I'll only have to change the backend, not all the code that accesses it.
--- Quote ---May I say how impressed I am with wincab so far, from downloading to adding music to mod'ing an existing skin to match the colours of my cab was done in about 45 mins. Was used extensively on Saturday night and even after I give a quick demo I normally get questions, how do I scroll down again, how do I add a track, ..., not one question - cool :applaud:
--- End quote ---
Glad you like it... thank you for the kind words!
ex_directory:
Hi Chris,
Have been thinking about this thread for a couple of days, also dug out my old source code hoping it may give me some inspiration...
--- Quote from: Chris on March 31, 2008, 12:23:15 pm ---That is essentially exactly how it is planned to work. The dialog only appears if you press BTN_FILTER_SELECT, which would then list all the predefined filters or allow you to create a filter on the fly. This is a replacement for a full-on search, which would require an full on-screen keyboard.
--- End quote ---
Thought you would be doing that kind of approach.
--- Quote from: Chris on March 31, 2008, 12:23:15 pm ---My "database solution" isn't even really a database. The DB file is loaded into memory verbatim as an array of strings. I have a bunch of helper routines written to access the data as if it were a database. This works because the data is for the most part read-only; the only data that changes is the play count and it only updates once per play, which is once every three to four minutes. Obviously this "solution" has serious scalability issues! In addition, the only real index is the song number. The path to the song is stored in a separate string array which is rebuilt on every run (that's what's happening when the library rescans on startup), unless the UpdateIndex option is set to False in which case it pre-loads the last path array and just trusts that all the songs are there and in the same place. (On an unrelated note, when the library is scanned, it is not looking for filenames; it looks at the last six digits of the file size and the first four characters of the filename and when it finds a match it assumes that is the same song. This allows a library built on DOS with 8.3 filenames to be compatible with a library built on Windows with long file names. That's why you don't see any file names if you open the database in a text editor.)
To support filters, essentially what I would do is at startup, check to see if either the database or filter definition has changed, and if so pre-scan the entire database to see what matches the filter. I would also do this for artists, genres and decades: create a separate index file for each possible genre, decade and artist (each one that exists at least once in the database) and pre-populate each with a list of integers representing the song ID's (position in the master string array) of each matching song. The ID3 specification allows for "artist" and "genre" to contain multiple values, so I have to account for that. None of this is spectacularly difficult, I just don't know if I want to manually build all the code for these indexes or just put in a proper database engine that will do all this for me and be more scaleable.
Ironically, the DB engine I'm looking at, SQLite, handles the database the same way I do: each record is a string and the entire database is held in RAM at once.
Either way I need to reorganize something, and once I start doing it I'm gonna break things. But all database access is virtualized through a couple of helper routines so I'll only have to change the backend, not all the code that accesses it.
--- End quote ---
Hmmm, I see your problems with this. Unfortunately in my program that I wrote I was not very innovative...
I essentially had the same thing - a large array of information loaded from file indexed by position in the array. For the most part I was using the whole array at once, but i then added functionality for which I needed to identify start points, if you like the start of chapters, so I did this by creating a separate integer array where each item in the array provided a number which was the start point of the chapter in the master array. The index of this array effectively the chapter number.
The next problem I had was I wanted to be able to view the data by categories which could be selected by the user and each category included a set of pages in the master array that were from several different chapters (I am using books as an analogy - actually had nothing to do with books!). I did this by creating an array per category and that array was simply of list of numbers that referred to the index in the original array again.
To make a further analogy back to wincab, if your database is sorted alphabetically (which I think it is) then your chapters would be letters of the alphabet. This gives you a quick jump to position when searching by artist - but not song.
The list of categories is the same as genres, i.e. the pages are from anywhere in the book, the songs are from anywhere in the database.
Of course I am not telling you anything here because what I have just described you have already said is your proposed solution above anyway! But what I suppose I am saying is that it worked for me, it was fast, relatively small code base and I was not reliant on any 3rd party db.
While not relevant to you as the db is based on user data, it meant I could ship a new single datafile if needed with ease (as all index arrays were created on startup) and could encrypt the datafile.
If it helps, I would say you need to base the design decision on the likely feature creep going forward, if you think this is going to open a door to more feature requests that you want to fulfil then maybe sqlite is the way to go now. Personally I would want to be sure the performance is there with a 3rd party db, it could bring a lot of overhead that you just don't need.
I hope this helps in some way :-\
ex_directory:
--- Quote from: Chris on March 29, 2008, 01:37:54 pm ---Also, the way the database is designed, using filters on a large library is slow; I need to either redesign my database to support indexes or put in a proper database backend like SQL. In fact, I do have the beginninghs of an SQL backend in place based on SQLite but I'm having trouble getting it to compile in DOS.
--- End quote ---
Had a bit of time so looked at the sqlite homepage, looks quite a neat solution, I wonder though how much a good database design is required to get maximum performance from this, it would seem the flatest design is likely to give the most optimal results. There is also a section on compiling for dos which I am sure you have seen here.
Above you say your proposal for using filters in memory will be slow, are you just referring to the startup procedure or the usage of wincab? I know it costs time but I wonder if a direct performance comparison would yield the sqlite solution to be quicker?
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version