Unable to scrape. Site checker shows no connection

[McSpud]

Hi
I've looked through the forum and can't find a solution.
I'm comparitively non techy with no Linux or Telnet experience.

I must have changed a setting somewhere or something but don't know where.
Although I can connect to internet in MSP when I use Site checker it's all red crosses.
I've done a rescan of all media and lost all artwork and info.
Tried total reinstall. Even put in a virgin Hard Drive and installed on to that. Still all red crosses... I don't have a clue whats wrong.
Currently put old hard drive in and moved one movie into Video ( having renamed original Video) and carried out a scan.
Here's the log but I don't understand it.
Don't hold back if i've done something stupid. I love Oversight cos It makes the popcorn hour idiot proof ( maybe not and look good.
Hope someone can help

[INFO] os version Linux version 2.6.15-sigma (ctliew@xeon4x) (gcc version 4.0.4) #3220 PREEMPT Mon Aug 16 14:13:22 MYT 2010
[INFO] nmt version 00-17-091216-15-POP-403
UNPAK=[/share/Apps/oversight/conf/unpak.cfg]
[INFO] 17:47:38 : $Id: catalog.awk 1425 2010-09-07 18:35:08Z lordylordy $
[INFO] 17:47:38 : Arg:[JOBID=12131]
[INFO] 17:47:38 : Arg:[PID=12131]
[INFO] 17:47:38 : Arg:[NOW=20101028174737]
[INFO] 17:47:38 : Arg:[DAY=Thu.PM]
[INFO] 17:47:38 : Arg:[START_DIR=/opt/sybhttpd/localhost.drives/HARD_DISK/Apps/oversight]
[INFO] 17:47:38 : Arg:[LS=/share/Apps/oversight/bin/nmt100/busybox ls]
[INFO] 17:47:38 : Arg:[APPDIR=/share/Apps/oversight]
[INFO] 17:47:38 : Arg:[CONF_FILE=/share/Apps/oversight/conf/catalog.cfg]
[INFO] 17:47:38 : Arg:[COUNTRY_FILE=/share/Apps/oversight/conf/country.txt]
[INFO] 17:47:38 : Arg:[DEFAULTS_FILE=/share/Apps/oversight/conf/.catalog.cfg.defaults]
[INFO] 17:47:38 : Arg:[LOG_TAG=]
[INFO] 17:47:38 : Arg:[PLOT_DB=/share/Apps/oversight/plot.db]
[INFO] 17:47:38 : Arg:[UNPAK_CFG=/share/Apps/oversight/conf/unpak.cfg]
[INFO] 17:47:38 : Arg:[UID=nmt]
[INFO] 17:47:38 : Arg:[OVERSIGHT_ID=nmt:nmt]
[INFO] 17:47:38 : Arg:[AWK=/share/Apps/oversight/bin/nmt100/gawk --re-interval ]
[INFO] 17:47:38 : Arg:[NMT_APP_DIR=/mnt/syb8634]
[INFO] 17:47:38 : Arg:[g_tmp_dir=/share/Apps/oversight/tmp/12131]
[INFO] 17:47:38 : Arg:[INDEX_DB=/share/Apps/oversight/index.db]
[INFO] 17:47:38 : Arg:[DEBUG1]
DBG = 1
DBG = 1
[INFO] 17:47:38 : Arg:[NOWRITE_NFO]
[INFO] 17:47:38 : Arg:[NEWSCAN]
[INFO] 17:47:38 : Arg:[GET_PORTRAITS]
[INFO] 17:47:38 : Arg:[GET_POSTERS]
[INFO] 17:47:38 : Arg:[GET_FANART]
[INFO] 17:47:38 : Arg:[/share/Video]
[INFO] 17:47:38 : Scan Path:[/share/Video]
[INFO] 17:47:38 : status:scanning
[INFO] 17:47:38 : load /share/Apps/oversight/conf/.catalog.cfg.defaults
[INFO] 17:47:38 : catalog_film_folder_fmt=[/share/Movies/{:TITLE:}{ (:YEAR}{ - :CERT:}]
[INFO] 17:47:38 : catalog_tv_file_fmt=[/share/Tv/{:TITLE:}{ - Season :SEASON:}/{:NAME:}]
[INFO] 17:47:38 : catalog_nfo_read=[yes]
[INFO] 17:47:38 : catalog_nfo_format=[xmbc]
[INFO] 17:47:38 : catalog_poster_prefix=[ovs_]
[INFO] 17:47:38 : catalog_format_tags=[bd,dd5,dts,cd1,cd2,720p,1080p,1080i,ac3,dvd,avchd,blu720p,blu1080p,bluray,brrip,​dsrip,img,hdtv,pal\>,ntsc\>,pdtv,r5,repost,repack,screener,workprint,ws\>,x264,xvid,yenc,par2,remaster,download,subtitle,720x,1280x,1920x]
[INFO] 17:47:38 : catalog_scan_paths=[/share/Video,/share/Movies,/share/Tv,/share/Complete]
[INFO] 17:47:38 : catalog_watch_paths=[NAS1,NAS2/movies]
[INFO] 17:47:38 : catalog_watch_minutes=[60]
[INFO] 17:47:38 : catalog_watch_torrents=[0]
[INFO] 17:47:38 : catalog_ignore_paths=[/share/~Other,*/Download]
[INFO] 17:47:38 : catalog_ignore_names=[*<samp>*,*<sample>*,divxfactory-*,nympho-*,fle-*,dfa-*,starlets-*,wde-*,swe6-*,ps-*,slieze-*,xcite-*,ichiban-*,motto-*]
[INFO] 17:47:38 : catalog_title_country_list=[USA,UK]
[INFO] 17:47:38 : catalog_cert_country_list=[gb,UK,us,USA,ie,Ireland]
[INFO] 17:47:38 : catalog_imdb_links_required=[4]
[INFO] 17:47:38 : catalog_imdb_titles_required=[4]
[INFO] 17:47:38 : catalog_user=[nmt]
[INFO] 17:47:38 : catalog_group=[nmt]
[INFO] 17:47:38 : catalog_search_engines=[YAHOO,YAHOOUK,YAHOOAU]
[INFO] 17:47:38 : catalog_tv_plugins=[THETVDB,TVRAGE]
[INFO] 17:47:38 : catalog_imdb_source=[desktop]
[INFO] 17:47:38 : catalog_movie_search_regex1=[^[-a-z0-9.]+$]
[INFO] 17:47:38 : catalog_movie_search_order1=[IMDBFIRST,ONLINE_NFO,IMDBLINKS]
[INFO] 17:47:38 : catalog_movie_search_regex2=[ .(20|19)[0-9][0-9]]
[INFO] 17:47:38 : catalog_movie_search_order2=[IMDBFIRST,ONLINE_NFO,IMDBLINKS]
[INFO] 17:47:38 : catalog_movie_search_regex3=[..........]
[INFO] 17:47:38 : catalog_movie_search_order3=[IMDBFIRST,ONLINE_NFO,IMDBLINKS]
[INFO] 17:47:38 : catalog_movie_search_regex4=[.]
[INFO] 17:47:38 : catalog_movie_search_order4=[IMDBFIRST,ONLINE_NFO,IMDBLINKS]
[INFO] 17:47:38 : catalog_config_version=[1]
[INFO] 17:47:38 : catalog_cache_film_info=[yes]
[INFO] 17:47:38 : catalog_genre=[Action,a,Adventure,e,Animation,i,Biography,b,Comedy,c,Crime,j,Documentary,k,Dram​a,d,Family,f,Fantasy,l,Film-Noir,o,Game-Show,g,History,p,Horror,h,Music,q,Musical,m,Mystery,u,News,n,Reality-TV,v,Romance,r,Sci-Fi,s,Short,x,Sport,y,Talk-Show,z,Thriller,t,War,w";]
[INFO] 17:47:38 : catalog_max_actors=[8]
[INFO] 17:47:38 : catalog_follow_symlinks=[0]
[INFO] 17:47:38 : load /share/Apps/oversight/conf/catalog.cfg
[INFO] 17:47:38 : Ignoring user setings for catalog_format_tags
[INFO] 17:47:38 : Overriding catalog_imdb_source: desktop -> mobile
[DEBUG] 17:47:38 : index check catalog_movie_search_regex1=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_regex1
[DEBUG] 17:47:38 : index check catalog_movie_search_order1=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_order1
[DEBUG] 17:47:38 : index check catalog_movie_search_regex2=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_regex2
[DEBUG] 17:47:38 : index check catalog_movie_search_order2=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_order2
[DEBUG] 17:47:38 : index check catalog_movie_search_regex3=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_regex3
[DEBUG] 17:47:38 : index check catalog_movie_search_order3=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_order3
[DEBUG] 17:47:38 : index check catalog_movie_search_regex4=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_regex4
[DEBUG] 17:47:38 : index check catalog_movie_search_order4=1
[INFO] 17:47:38 : Ignoring user setings for catalog_movie_search_order4
[INFO] 17:47:38 : load /share/Apps/oversight/conf/country.txt
[INFO] 17:47:38 : ignore path = [^(/share/~Other|.*/Download)]
[DEBUG] 17:47:38 : RENAME_TV=
[DEBUG] 17:47:38 : RENAME_FILM=
[DEBUG] 17:47:38 : catalog_format_tags=(\<|_)(bd|dd5|dts|cd1|cd2|720p|1080p|1080i|ac3|dvd|avchd|blu720p|blu1080p|bluray​|brrip|dsrip|img|hdtv|pal(\>|_)|ntsc(\>|_)|pdtv|r5|repost|repack|screener|workprint|ws(\>|_)|x264|xvid|yenc|par2|remaster|download|subtitle|720x|1280x|1920x)
[INFO] 17:47:38 : >Begin scan_page_for_matches[http://search.yahoo.com/search?ei=UTF-8;eo=UTF-8;p=test]
[INFO] 17:47:38 : [/search][http://[-_.a-z0-9/]+/search\>][0]
[INFO] 17:47:44 : Deleting '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz'
[DEBUG] 17:47:44 : WGET ['http://search.yahoo.com/search?ei=UTF-8;eo=UTF-8;p=test']
[DEBUG] 17:47:44 : SYSTEM : [[wget -O '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz' -U "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040613 Firefox/0.8.0+" -T 30 -t 2 -w 2 -q --no-check-certificate --ignore-length --referer="http://search.yahoo.com/" --header="Accept-Encoding: gzip" -c 'http://search.yahoo.com/search?ei=UTF-8;eo=UTF-8;p=test' && ( gunzip -c '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz' || gzip -c -d '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz' || cat '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz') > '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp' 2>/dev/null && rm '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz']]
[ERR] 17:48:14 : Return code 1 executing wget -O '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz' -U "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040613 Firefox/0.8.0+" -T 30 -t 2 -w 2 -q --no-check-certificate --ignore-length --referer="http://search.yahoo.com/" --header="Accept-Encoding: gzip" -c 'http://search.yahoo.com/search?ei=UTF-8;eo=UTF-8;p=test' && ( gunzip -c '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz' || gzip -c -d '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz' || cat '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz') > '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp' 2>/dev/null && rm '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz'
[INFO] 17:48:14 : Deleting '/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match.tmp.gz'
[ERR] 17:48:14 : getUrl:scan4match: Failed getting [http://search.yahoo.com/search?ei=UTF-8;eo=UTF-8;p=test] into [/share/Apps/oversight/tmp/12131/catalog.12131.__scan4match]
[INFO] 17:48:14 : <End scan_page_for_matches[http://search.yahoo.com/search?ei=UTF-8;eo=UTF-8;p=test]=[=[0]]
[DEBUG] 17:48:14 : post best:<empty>
[INFO] 17:48:14 : bestScore =
[INFO] 17:48:14 : remapping http://search.yahoo.com/search?ei=UTF-8;...o=UTF-8;p=
[INFO] 17:48:14 : >Begin scan_page_for_matches[http://ask.com/web?q=test]
[INFO] 17:48:14 : [/web][http://[-_.a-z0-9/]+/web\>][0]
[INFO] 17:48:19 : Deleting '/share/Apps/oversight/tmp/12131/catalog.12131.1__scan4match.tmp.gz'
[DEBUG] 17:48:19 : WGET ['http://ask.com/web?q=test']
[DEBUG] 17:48:19 : SYSTEM : [[wget -O '/share/Apps/oversight/tmp/12131/catalog.12131.1__scan4match.tmp.gz' -U "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040613 Firefox/0.8.0+" -T 30 -t 2 -w 2 -q --no-check-certificate --ignore-length --referer="http://ask.com/" --header="Accept-Encoding: gzip" -c 'http://ask.com/web?q=test' && ( gunzip -c '/share/Apps/oversight/tmp/12131/catalog.12131.1__scan4match.tmp.gz' || gzip -c -d '/share/Apps/oversight/tmp/12131/catalog.12131.1__scan4match.tmp.gz' || cat '/share/Apps/oversight/tmp/12131/catalog.12131.1__scan4match.tmp.gz') > '/share/Apps/oversight/tmp/12131/catalog.12131.1__scan4match.tmp' 2>/dev/null && rm '/share/Apps/oversight/tmp/12131/catalog.12131.1__scan4match.tmp.gz']]

[lordy]

Hi, wget is taking more than 30 seconds. There is a weird DNS bug on NMT platform that causes this.
Sometimes the dNS query takes 30 seconds before successful completion, but unfortunately I have wget timeout at 30 seconds with a failure.

Next version of Oversight will timeout after 31 seconds, so hopefully fix this.

In the mean time the simplest option is to change DNS server.
http://code.google.com/p/oversight/wiki/...lways_fail

I'd be interested to know if you changed anything on you network just before this started happening.

[McSpud]

WOW.. that was fast. I was tinkering with things. I wanted to get into Transmission to increase the number of shares so did try to Telnet unsucessfuly. Found a nice GUI that let me do it after that. also added torrentwatch. Don't think I could have done any damage but who knows. Will look at the link you sent.
Cheers

[McSpud]

Can someone tell me if its possible to access the nmt to carry out the info in the above link from a Windows 7 machine . and if it's possible what I need to do it.
Cheers

PS It couldn't be beacuse I changed my router could it.??

[lordy]

(10-30-2010 03:01 PM)McSpud Wrote:  Can someone tell me if its possible to access the nmt to carry out the info in the above link from a Windows 7 machine . and if it's possible what I need to do it.
Cheers

You can make the changes via the standard NMT setup screen.
Go to Setup->Network->Configure IP Then set
DNS Setup = Manual
And then set Primary and Secondary DNS to values on the page and retest.

Quote:PS It couldn't be beacuse I changed my router could it.??

If it worked before and stopped working after then probably. That does count as "changing something on your network"

[McSpud]

Lordy .. You are a STAR
I am the classic example of why a little knowledge is dangerous.
I'd mucked about with DNS setings and that was why it wasn't scraping. Cheers for pointing out something so simple that i'd mucked up.
Just gonna hit that donate button
Cheers