ÆON Movie Wall / Eversion for Oversight - Beta Version - Released

[vaidyasr]

Re-uploaded the jukebox again due to a typo in the build.php which does not allow to build aeon skin.
The season names are showing in the episode page only the first word. Don't know the reason. Need to fix it.

@accident, the episode title should be encoded with %20 without spaces?. Because if the episode name is 'The first movie', it shows only The

[accident]

most of the yamj data is just xml safe encoding. display text should be xml entity encoded, there is a php function set to xml1 to do this for you. http://www.php.net/manual/en/function.ht...php#106535 is a direct post of someone using it for this.

Code:

& replace with &
< replace with &lt;
> replace with &gt;

Filenames there are 2 different encodings, one of those is mega confusing to follow the java for but many sections of yamj have been changed to an easier encoding. eversion has a function to allow a skinner to mimic this if needed so it would be a good way to go and we don't need to worry about how yamj does it because all the data is made outside of yamj.

The encoding is just converting all of the following to a - (dash).

Code:

/\?*:<>

That encoding should be used in filenames AND the xml nodes that including that filename. Most handy if you have filenames made from titles.. like Set_tv show name_#.. you would filename safe that tv show name part and all existing eskins are expecting that format (and naming for there special things).

Dont' forget to also utf8_encode the filenames.

Some XML has an attribute called encoded/ In that case, only the filename encoding is used in the encoded attribute and the other attribute is just xml friendly encoded (backwords compatible, encoded added later). This is mainly used in the index section.

For urls, a standard URI encoding is used. the only real spot your need to have in the xml is the fileURL for playback. This is hte %20 style escape encoding, BUT the uri variation does not encode : / . - or _. (might eb some others).. it's a standard encoding method know as URI Encoding so if php doesn't have a function, it'll be fast to find it.. IIRC I think an old php script I have uses a large class with all the ajax encoding methods including this if you want me to look for it. sample php code to ajax talk to google api or similar should have something also as that's the same encoding for that.

I added something similar to eversion for oversight, but I escape it then convert the few characters back.. not pretty but good enough to make special characters play right.

---

You posted that you changed the download but I only see 1.0.1 download. did you call it the same filename?

[vaidyasr]

Yes, same filename.

[accident]

Missing when running the new build:

Western, Adult, Reality, Science, Fiction, Special, Interest. (I think the last 4 are just 2 genres with spaces between words).

this is going to take about 3 days to finish.. with about 10,000 items int he library according the scan count when it was scanning, your processing at the speed of about 2 minutes per xml file.. oversight scanned these at a speed of about 2 seconds per item with another 10 seconds for the artwork part.

EDITED TO UPDATE: it actually never finished the first one.. still waiting.. I guessed 2 minutes but I'm very wrong, its much longer...

I'm on the 300 so let me copy the data in case this is just some differences in a 300 running it since I'm seeing other slow down quirks with nmt apps..

[Slevin]

The new upload still doesn't seem to be working:

Code:

/opt/sybhttpd/localhost.drives/HARD_DISK/Apps/JukeBox/webui # /share/Apps/lighttpd/bin/php-cgi build.php
X-Powered-By: PHP/5.2.8
Content-type: text/html

/opt/sybhttpd/localhost.drives/HARD_DISK/Apps/JukeBox/webui #

[darucla]

Checking a couple of HTML files, they seem all to be showing 29.97fps for (UK) TV files, which should be 25fps.


Using the revised 1,0.1 version, still insanely slow scanning on my A-200. Firmware 02-04-110519-21-POP-411 (02-04-101104-21-POP-411) (27.05.2011)

[accident]

I made a copy of the entire oversight and JukeBox app folders onto an ubuntu 11 server headless amd-e 350 desktop. This is my test box in the house, dual core, openCL enabled install. whatever the current php version via the up to date lamp base install is. There's also 8GB of memory with about 7GB free.

I changed a couple paths in the build.php script so it would run, just the oversight/ paths to the full local path to the oversight folder did the trick.

my command line was: php build.php eversion

The 10,000+ items as reported by the overscan scanner is making xml files at the rate of 1 per 2-3 minutes.

I modded your script for speed in the following way:

I moved these lines from buildMovieDetailsXML to just after the ksort before you start processing:
$plot = readPlotData();
$people = readPeopleData();

I then in buildMovieDetailsXML added them as globals:

global $people,$plot;

I added some debug printing and and it quickly went over teh indexes and then over 3 minutes loaded up all the plots, people and movies, sorted teh movie list and began making xml files..

I made ALL xml detail files in 4 second, index files took 1 second per file. total processing time with this mod was approximately 4 minutes.

I am about to run the modded build.php on the 300 to see what happens there..

---

UPDATE:

took the script, fixed the paths and ran it on the 300.. took about 10 minutes to read in the initial data and 3-4 minutes to make all the detail xml files.. index files took about 15 seconds per file.. total run time is probably going to be about an hour on an a300. not thrilling but usable.. of course your maxing hte cpu at 100% and there's no memory left.

Speed improvement:
- your making categories.xml everytime you make each index file. just make the file once and use a copy of the data to add in teh current data for each index file. no need to remake the file from scratch the data doesn't change and you just replace data for current anyway.
- nothing else is standing out as reprocessing the same thing over and over and over again.

Eversion improvements:
- 20 movies per index instead of 27 would be better. 3 rows can remain but we only need 1 row, we don't use the rows for formatting
- write out Categories.xml normally. For the top of index file, only make the current node but it needs the normal xml format

I'm starting to review the xml output now, I'll be back with comments on that.

[mike7]

(03-12-2012 05:56 PM)vaidyasr Wrote:  Re-uploaded the jukebox again due to a typo in the build.php which does not allow to build aeon skin.
The season names are showing in the episode page only the first word. Don't know the reason. Need to fix it.

I have plot info in Russian, in version 1.0.1 plot info not encoded in UTF-8, so I can see only question marks instead characters.

Previous version didn't have such issue, but xmls for Categories, years etc were generated with wrong encoding, as result there were no corresponding htmls.

[jonny555]

still carnt build html/xml with command line
# cd /share/Apps/JukeBox/webui
# /nmt/apps/server/php build.php
updated version of jukebox installed through csi jukebox in web services
but comes up with return error c200

[accident]

I can provide yamj xml files and your xml files for comparison so let me know what senarios you need examples for and I'll zip them up for you.

General running:

as new tv episodes are added to a tv show in oversight, your going to end up skipping the xml updates because the xml will be there already.

XML you wrote Analysis

filenames:

You don't need to use yamj naming for details files, you should write whatever is easiest to find when your checking it again later. Some of your names I could see overlapping in the future but I don't see any, but it's hard to tell with 7000 xml files to look at.

Yamj naming for index files is very important as a lot of skin logic in all skins does use the naming.

Filenames are not utf-8 encoded so unicode support is broken and eversion can't read them all in. gaya may have html filename issues also, didn't test aeon.

xml files are utf-8 encoded now, that looks good.



If you don't have data to write, don't make a blank node, put in UNKNOWN.
ex:

PHP Code:

you:
      <audioCodec></audioCodec>
      <audioChannels></audioChannels>

      yamj:
      <audioCodec>UNKNOWN</audioCodec>
      <audioChannels>UNKNOWN</audioChannels> 


Sets

I'm not sure what your set support is but I have a lot of weird sets with movies and tv shows mixed together that wouldn't be in a set. Can you explain what your support is and I'll comment on where it went wrong.

As far as I can tell initially, all sets were made with exactly the same content..

tv shows:

Detail xml file:

You wrote 1 file for the entire show no matter how many seasons there is. YAMJ would make 1 xml file per season. aeon would get each file transformed to make 1 html file per season. ex: a show with 4 seasons, there would 4 xml files and aeon would make 4 html files.

Your current xml file output is perfect for an index, more on this is a moment because I need to explain sets and explain how this needs slight modificiation.

Sets:

completely wrong and all sets ahve the same data in them.. So let me just bullet point how to create a tv show xml setup..

1: create the detail xml your make now, change the details filename to be Set_tvshowname_1.html, not sure about isSet, I'll have to check
2: make 1 xml file per season. only the videos for each season should be in it, make sure the season name is correct. for aeon, use these files to make html files. if oversight has per season artwork, make sure artwork is correct for each season
3: make a Set_tvshowname_#.xml file. This is an index with all of the files from #2 sorted in season order. Transform these with index for aeon to make a Set_tvshowname_#.html file
4: When making the other index files, use the xml file from #1 for the tv show.

set_tvshowname_# needs to be filename safe encoding and utf8_encode(). See previous posts on what yamj does for filename safe encoding.

Newer yamj data:

theres some new fields for airdate and such, I'll work up an example for you.

Incorrect:
fileparts for multipart episodes (multiple episodes, 1 file) is completely wrong. I'll have to get you an example


Movies:

multipart movies do not have all files in the xml

All:
plot and outline are not the same thing. if tagline and outline is not supported in oversight, just make them UNKNOWN

iso files have vod='' it should not have this and it's missing vcd='2'

I do not have any bdmv, bdav, or video_ts in this scan to make sure the same vcd='2' is there.

artwork paths look funny now.. looks like ../fullpath to oversight location. use the old way of ../oversight that worked well, I was incorrect saying it was a problem.

& symbols are not encoded as & all xml is broken (although flash doesn't complain, xslt will complain)

fileurl network share paths are wrong.. standard for oversight but aeon will not be able to play anything. eversion already fixes this itself as well as handles the mounting of shares.. aeon can't do the mounting so don't worry there.

fileurl are not uriencoded, eversion handles this itself, might be problems with aeon and

Missing for aeon:

I think the version of aeon your using uses the playlist file. There is jsp playlist files made by yamj.. I don't know much abotu these, you'll want examples from a real html skin user. make sure you get all the episode examples because it makes different jsp files for each episode. eversion so not use or need these.


SUPPORT FOR ANY HTML SKIN:

the way you xsl transform, you could use your java compile + saxxonhe to transform any yamj skin with these files using exactly the same xslt processor yamj does so no skin converting.

[Slevin]

hmmm.. creating xml files works using

Code:

# /share/Apps/lighttpd/bin/php-cgi /share/Apps/JukeBox/webui/build.php eversion

html generation using does not

Code:

# /share/Apps/lighttpd/bin/php-cgi /share/Apps/JukeBox/webui/build.php aeon

(it should work without the " aeon" too right??

Probably as accident mentioned in the post above?

(03-12-2012 08:53 PM)accident Wrote:  & symbols are not encoded as & all xml is broken (although flash doesn't complain, xslt will complain)

[accident]

(03-12-2012 09:13 PM)Slevin Wrote:  hmmm.. creating xml files works using

Code:

# /share/Apps/lighttpd/bin/php-cgi /share/Apps/JukeBox/webui/build.php eversion

for just xml and eversion:

- pch with nmt installed
- install oversight and scan
- install jukebox (see below build command line) (only install jukebox, ignore all instrcutions, just download and csi install)
- install eversion
- run eversion as via yamj app or clicking eversion.phf in /Apps/Eversion when drive navigating

Note: current version has some quirks, you may not get artwork and sets is really off.

to build:

Code:

cd /share/Apps/JukeBox/webui
/nmt/apps/server/php build.php eversion

[accident]

I just noticed that Categories.xml is written wrong.. seems everything is missing from the file, it's correct in the index files though

[vaidyasr]

(03-12-2012 08:00 PM)mike7 Wrote:  Previous version didn't have such issue, but xmls for Categories, years etc were generated with wrong encoding, as result there were no corresponding htmls.

If possible, please provide the sample data. I found the actor/director name was not showing properly, so I did a utf8 encode to store in xml. If you can provide the db (no need of images, only index.db, plot.db & actors.db), I'll test it and fix it.

---

(03-12-2012 07:17 PM)accident Wrote:  The 10,000+ items as reported by the overscan scanner is making xml files at the rate of 1 per 2-3 minutes.

It took almost 3-4 seconds per xml and html with the sample data. Possibly I have only 350+ movies. I'll fix the code for optimizing the speed as you suggested.

[vaidyasr]

Optimized the code and now it takes in the C200 itself 1m10seconds for 70 movies including building xml & html. Some of them are TV shows, which contains multiple episodes.

---

Fixes:
plot and outline are not the same thing. if tagline and outline is not supported in oversight, just make them UNKNOWN - Done
iso files have vod='' it should not have this and it's missing vcd='2' - Done
artwork paths look funny now.. looks like ../fullpath to oversight location. use the old way of ../oversight that worked well, I was incorrect saying it was a problem. The image path haven't changed from last version. The only check was whether the image exists or not. If not, it will use the AEON default dummy image.
& symbols are not encoded as & all xml is broken (although flash doesn't complain, xslt will complain) - File title with special characters already encoded in 1.0.1, and I found no issue with xsltproc.
fileurl network share paths are wrong.. standard for oversight but aeon will not be able to play anything. eversion already fixes this itself as well as handles the mounting of shares.. aeon can't do the mounting so don't worry there. Done
fileurl are not uriencoded, eversion handles this itself, might be problems with aeon and - Done