[NewzBook] 1.2.7beta Newzbin Bookmark + Mobile Search Utility + NZBget interface

[joshua.lyon]

NewzBook 1.2.7beta Newzbin Bookmark + Mobile Search Utility + NZBGet interface
NewzBook Major Features:

• Monitor and automatically download Newzbin.com bookmarks
• NZBMatrix Integration - search and download (no bookmark manager)
• NZBget Micro Application - view and control your usenet downloads
• Newzbin.com Search - lightweight search from anywhere
• Filemanager - view your files from anywhere
• Filemanager - Remotely start files on the PCH
• File Manager - Remotely control the PCH
• Security - secure external access
• Media Player - web based media player
• Basic Skin System
• Secure internet access
  

NewzBook is available via CSI from the Applications tab. Or download v1.2.7beta Here for manual install

Note: NewzBook 1.2.7 was updated from my XBMC Linux box. I did not test the changes on a PCH as I do not have one hooked up. Install at your own risk.

• Click here to see a video preview
• Click here for an iPhone video simulation
• Click here to see the file manager features
• Click here to see the MediaPlayer
• See below for screenshots...
  

FAQ
How do I submit a bug/issue?
First, please check the Active Issues and Issues to Verify lists. If the issue is on the Active Issues, I will get to it in a future version. If the issue is on the Issues to Verify list, it has either been released in a recent version or will get released soon.

If your issue does not appear to exist in either of those lists, please post the issue on this thread and make sure you include as much information about the issue as possible. At the very least, please include:
*Version of NewzBook [1.0.1beta, 1.2.4beta]
*Type of Hardware [A110, C200]
*Steps to reproduce the problem
*What happened? And what did you expect to happen?
*Other actions performed around the same time [Reboot PCH, Installed App XXX, etc]

What is the URL for the NewzBook application?
http://popcornhour:9999/NewzBook_web/
You may have to replace popcornhour with the name or IP address of your NMT.

How often does NewzBook check for new bookmarks?
NewzBook checks for new bookmarks every 30 minutes.

Can I change how often NewzBook automatically checks for bookmarks?
Easy: At this point in time, there is no user-friendly way to do so.
Moderate-Hard: You can modify the appinfo.json and cronchecker.sh files. Look for the "30" and change it. For example to check every ten minutes, use 10. Be VERY careful! After changing the files, you will need to restart NewzBook from the "Installed" tab of CSI.
Hard: If you are technically savvy, you can telnet in and modify the crontab entry.

Can NewzBook queue up a download from a ReportID?
Yes. NewzBook can take a Newzbin ID and add it directly to the download queue. It is the bottom option on the web-interface called Download NZB via Report ID. This field can also take a newzbin report URL or newzbin NZB URL.


How do I check the logs?
From the web interface, choose the View Log for Automatic Job option. Please note that the automatic job runs every 30 minutes. There will either be a green success message or red warning message at the bottom indicating if the process appears to be running properly. You must wait at least 30 minutes from the initial install / power-on to verify that the automatic job is running properly.

A new log is created for each week. The viewer shows the latest log (eg. this weeks log). Old logs are kept in the logs folder of the app for 30 days before being deleted.

How can I tell if NewzBook is working?
You can either check the logs using the above method or run a test using the web interface. The web-interface provides a number of tools for testing.

What does View Bookmarks do?
This option gets the list of your bookmarks from Newzbin and displays them on the screen. It also provides an option to queue the download in your Usenet client.

What does Download Bookmarks Now do?
This option attempts to download all the bookmarks in your Newzbin bookmark queue. You are limited to 5 NZB downloads within a 1 minute period per IP due to Newzbin rules. If you would like to download more, simply wait a minute and try the option again to download another batch of 5 NZBs. Or let NewzBook automatically do the work for you when it checks every 30 minutes.

How is NewzBook licensed?
GNU GPL v3

How do I manually install a pre-release version?
Occasionally, I will post newer version of the app to the google code repository before releasing the app to the general public. Use the following instructions to manually install a "pre-release" version. Download the zip file from the above link then... Open CSI. Click File > Install from File. Click open and select the zip file you downloaded. Click next to go to the confirmation page. Click next to install. If everything goes right you should see a success page. Click next. NewzBook should now be installed.

Can I install this on a platform other than the NMT / Popcornhour?
Sure. Check out the wiki entry:
http://code.google.com/p/newzbook/wiki/OptwareInstall

Will you send me a Newzbin Invite?
I do not send out Newzbin invitations. Be aware that people who send out invitations are partially responsible for your actions so most people will be hesitant about giving out invitations.

Changelog:

Code:

1.2.7beta:
*Includes search support for NZBMatrix
*Includes NZB download support for NZBMatrix
*Started work on SABnzbd integration -- VERY ALPHA.
*Added stripslashes to fileman to support other platforms

1.2.6beta:
*Media Player - web based media player added
*Media Player - playlist management system
*Media Player - Core AJAX functions
*Skin System - basic skinning (CSS) system added
*Skin System - regular and mobile specific skins
*Major code moved into functions (still more work to be done here)

1.2.5beta:
*File Manager added - view what files are on your device
*File Manager - play MP3s and view pictures remotely (eg. on your laptop or from work)
*File Manager - remotely play files on the PCH
*File Manager - remotely control the PCH (start, stop, FFW, etc)
*File Manager - view files (as text) via the web
*File Manager - Icons based on file type (eg. video, picture, music)
*Security - internet security
*Security - internet hash (for 3P apps and remote play of songs)
*Memory Limit Fix - large NZBs can be downloaded without issue now
*NZBGet - fixed bug where paused files were hidden
*There are probably others, but this was a big release and I didn't track all of them...

1.2.4beta:
* Added some code to automatically check cron job and add it if it doesn't exist. This is a workaround until CSI automatically fixes cron jobs upon reboot.

1.2.3beta:
*Default configurations selections (buttons) A110, C200, CSI
*Configuration warning messages if NewzBook detects an invalid configuration
*Zip based Log export
*Log validity check - warning if job isn't running
*NZBget rate control - nothing special, but it was missing
*Conditional check for search content (eg. NFO link doesn't show if it's unavailable)
*Auto failover of cronjob for C200 / Optware users
*Basic Debugging page added (can be accessed from configuration page)

1.2.2beta:
*NZBget web interface added
*Configuration moved to it's own page
*Basic logic added for if nzbget interface is enabled or not

1.1.0beta:
*Newzbin search now included - mobile friendly!
*loading image added
*language/text on interface cleaned up
*save file messages made more accurate
*Newzbin report URLs can be used in download box

1.0.2beta:
    * fixed  issue #2  where filesizes greater than 2 gb were reported as 2 gb.
    * fixed  issue #1  where bookmarks with a colon in the name had the name truncated where the colon was.

1.0.1beta:
    * First packaged / distributed release
    * Set the application to stop if the username and password aren't configured
    * Pull the header information for a bookmark check and display the error to the user if necessary

1.0.0beta
    * User friendly error messages implemented
    * Use DirectNZB header information in application
    * cronjob created with warning messages

pre-beta
    * Core functionality built
    * Basic web interface created

[hugbug]

Great work!
I'm glad to see a complete and automated solution for newzbin, although I don't use it myself.

Could you possibly make a How-to for a manual install to use it on other devices?

You didn't mention the license. I think I see a part of nzbgetweb there, so it's "GPL v. 2 or later" I guess.

I see a hardcoded address "http://ip/NewzBook_web/index.php" in "cronjob.php". is it OK?

BTW, instead of using cron the script could be called via nzbget's scheduler. That has an advantage, that all printed messages will be seen in nzbget' log.

[joshua.lyon]

Thanks for the comments, hugbug!

* I updated the code to remove the hardcoded address. It's just used during a post-request to newzbin so it's really more of a placeholder than anything.
* I added the license info to the initial post: GNU GPL v3 (also indicated on google code) (Also, I'm not using any code from NZBgetweb at the moment.)
* That's an interesting thought that the application could be called from NZBget's scheduler. At this point, I didn't want any dependencies on other apps but I might tinker with it later.
* Do you know of other people who are interested in using it on other devices? What kind of devices did you have in mind? I could write something up as it's a fairly simple application to deploy -just some PHP code and shell script that can be run as a cron job.

If anyone else has any comments or questions - let me know.

[lordy]

Nice work

[hugbug]

(09-22-2009 06:15 PM)joshua.lyon Wrote:  * I added the license info to the initial post: GNU GPL v3 (also indicated on google code) (Also, I'm not using any code from NZBgetweb at the moment.)

I didn't realize the download link goes to googlecode, where the license is indicated .

(09-22-2009 06:15 PM)joshua.lyon Wrote:  * That's an interesting thought that the application could be called from NZBget's scheduler. At this point, I didn't want any dependencies on other apps but I might tinker with it later.

A good point, less dependencies is always better. May be as an option someday.

(09-22-2009 06:15 PM)joshua.lyon Wrote:  * Do you know of other people who are interested in using it on other devices? What kind of devices did you have in mind?

NZBGet is widely used on OPTWARE platform, which is available for many NAS, routers etc. Just a few tips for installing on a typical Linux system would be helpful, assuming the system already has a working web-server with php.

I've posted a link to this thread on NZBGet forum.

[mikemav]

Hi, great work Josh, I'll have to give this a go sometime. The only thing holding me back may be my anonymity paranoia- now I have a paid anon proxy server on my PC that's running newzbin DLs* since I prefer my real IP not be out there in public. Yes I know newsgroups are safer than torrents, etc, but I may have to try some kind of forwarding the NMT's internet connection through a PC shared internet connection w/ this proxy running just to make sure. Also this reminds me, can the combo of your NewzBook and NZBGet on the NMT connect to usenet via SSH? I use encryption now on my PC newsgroups account...

*this is a work of interactive fiction and theoretical practices I'm commenting on- I really download no files myself, and especially am careful to never violate any copyright laws personally of course, for the record...

[lordy]

Not quite sure what you are paying for there? Speaking strictly in terms of the Anonymity , and technology and not about content:

IMO you have decreased you anonymity and paid for the privilege, (by given up you details to two parties instead of one)

With just SSL Newgroups the usenet provider has all your details and snoopers can see your IP - if they are lucky and know what they are doing.

With SSL Newsgroups + Your Anon Proxy - The usenet provider still has your details and now an anon proxy also has your details. Snoopers can still find your IP at the proxy. (again if they are lucky and also technical),
The connection to newsgroups still uses your username - for Usenet Provider logs.

If you were trying to keep you anonymity from the Usernet server that would be OK, but they already know you. (I'm assuming its paid newsgroups because you have SSL access). I wont take this thread anymore off topic though..

[joshua.lyon]

I have not added or removed any security from the way you would typically download your NZBs from Newzbin in the NewzBook application. The application is using an API Newzbin provides. It's essentially the same level of security as if you were to manually log on to Newzbin and download your bookmarks... but instead the application does it automatically.

I understand the paranoia... NZBget does provide SSL capabilities for downloading so you can at least use that. If Newzbin ever does offer an SSL connection (and API), I'll be sure to implement it.

[hugbug]

(09-23-2009 08:23 PM)joshua.lyon Wrote:  If Newzbin ever does offer an SSL connection (and API), I'll be sure to implement it.

I think it does.

[joshua.lyon]

(09-23-2009 09:33 PM)hugbug Wrote:  I think it does.

Interesting. There is no talk about SSL or encryption anywhere in their documentation except for the implementation of it for their payment processing. I tried to access the site with the https and it did find a valid SSL certificate. I'll try a branch of code that makes an SSL post and see how it works out.

[ironjaw]

Josh, this little app you've written - its the Holy Grail for me! I currently use nzbget on my Buffalo Linkstation, via Optware. Also use Newzbin for usenet indexing and have been trying to find a way to update nzbs to the nzbget queue remotely. With your awesome skills, this may now be possible!!!

Thankyou for all the time and effort you've put in so far, I'm no unix guru, just a follower who rides on the coat-tails of the gifted!

A how-to for us optware users would be a great help!

Thanks again!

[joshua.lyon]

1.0.2 beta is now released.

Code:

1.0.2beta:
    * fixed  issue #2  where filesizes greater than 2 gb were reported as 2 gb.
    * fixed  issue #1  where bookmarks with a colon in the name had the name truncated where the colon was.

I also noticed that there was a capitalization error with the repository file that was preventing the file from being installed using the CSI repository. This is now fixed as well.

I've also added a wiki page for optware users. It's really slim right now, but I don't really know what other details those users need. If you have any questions post it here and if it's relevant I'll add it to the wiki.
Wiki Page for Optware Users

[ironjaw]

Thanks Josh, seems I have some tinkering to do over the weekend!

[ironjaw]

Hey Josh, regarding running this on Optware...

This is working fine for me on a Buffalo LS Live - all except the crontab script. I'm no unix expert, but I've managed to trace it down to an error when the script is being run from root. I can run the "php -q /yourpath/NewzBook/webui/cronjob.php" command manually from within the configuration/ folder, but nowhere else. So I think I've set the cronjob OK, just when it tries to run it, it can't. The error I'm getting is:

PHP Warning: parse_ini_file(../configuration/configuration.ini): failed to open stream: No such file or Directory in /opt/share/www/nzbook/webui/class.iniparser.php on line 20
[WARNING] Username and/or password are not set.

Any ideas? Is it a problem with me running php -q without the full path? I noticed in your NMT example it has a full path to the PHP interpreter. I'll see if I can dig out any info on the full path for Optware.

Once again, many thanks for the work and effort, this little app will save me hours of waiting around for stuff to finish,I can kick off jobs from work before I leave, so while I sit in traffic everything is ticking away

--------------------EDIT----------------------

Turns out it is /opt/bin/php-fcgi

Everything is working fine now!!! I'm so chuffed!!!!!

THANKYOU!

[mikemav]

(09-23-2009 06:49 PM)lordy Wrote:  Not quite sure what you are paying for there? Speaking strictly in terms of the Anonymity , and technology and not about content:

IMO you have decreased you anonymity and paid for the privilege, (by given up you details to two parties instead of one)

With just SSL Newgroups the usenet provider has all your details and snoopers can see your IP - if they are lucky and know what they are doing.

With SSL Newsgroups + Your Anon Proxy - The usenet provider still has your details and now an anon proxy also has your details. Snoopers can still find your IP at the proxy. (again if they are lucky and also technical),
The connection to newsgroups still uses your username - for Usenet Provider logs.

If you were trying to keep you anonymity from the Usernet server that would be OK, but they already know you. (I'm assuming its paid newsgroups because you have SSL access). I wont take this thread anymore off topic though..

Good points, and don't mean to go off topic- it was just that anonymity plays a part in my decision to use an NMT app or something on a PC, as I'm sure it does for many considering using torrents or usenet.

Anyway, my thinking was that, A) my usenet provider does not in fact know who I am. Prepaid Visa credit cards are available in any grocer in the States and work for many online transactions such as monthly usenet fees. Sign up from a public hot spot sometime and pay for with prepaid Visa and you're as close to a ghost as you can be I'd guess. ISP knows your doing something with news groups (over SSL) but that's it, and usenet provider sees a proxy IP if I'm running anon software. So it would be nice if Anonamizer or TOR or something would run on the NMT, but that is unlikely, so I wonder if I can just run Anonamizer for my PC internet access and forward that network access to the NMT (using the PC as a proxy for the NMT to connect thru?) But if I do that I might as well use Sabnzb+ on the PC to get NZBs as well.

So I think it's either live without the anon proxy and get convenience of not needing a PC on, or keep using the PC for all, unless I'm missing another way here...

Anyway, sorry if that is off topic. Great work on the newzbin bookmark utility!