chown -R HARD_DRIVE gets launched at each boot and can cause very long bootup time

[lundman]

Seems to me that it only happens if they top level directory of the mount is owned by root. If so, it - and everything under - is chowned to nmt. Then it does not happen the next time.

If yours always does it, try to figure out why the root directory is still owned by root.

[laureck]

(11-05-2009 11:46 PM)lundman Wrote:  Seems to me that it only happens if they top level directory of the mount is owned by root. If so, it - and everything under - is chowned to nmt. Then it does not happen the next time.

If yours always does it, try to figure out why the root directory is still owned by root.

Hello Lundman,
this is a good question (and one question I did not fail to ask myself, see the update part at the end of post #6)
I believe that the internal hard drive is mounted by the binary /bin/hdpart in S99miscinit startup script.
I originally thought that internal hard drive has to be mounted with uid and gid options so that its mounting point belong to nmt (/bin/hdpart does not use these options, you can tell this by issuing a 'mount' on a telnet session) but I tried to umount remount my external hard drive and saw that the mounting point inherits the filesystem '.' ownership so I do not understand why the recursive chown is triggered.

Maybe is the chown triggered elsewhere...

By the way, could you please try if you can reproduce this problem ?
Just 'touch' a new file in a subdirectory using a telnet session, verify that it belongs to root, reboot and see if ownership has changed.
I'd really appreciate some confirmation on this point.

[laureck]

Mount commands seem to lack the necessary uid,gid options after all

After several readings (mount man page, forums) it seems that the mounting point of a filesystem is owned by root:root when no uid and gid options are given.

Every one with a telnet session can verify that internal hard drive was mounted without these options (simply issue "mount" to get the mounted filessytems along with the mounting options).
Same applies to the external hard drives by the way.

Now assuming the mounting point is owned by root in its initial state a 'chown -R nmt:nmt' is performed on the whole hard drive at line 29 of /etc/init_nmt script.


So why does unmounting / mounting of the external hard drive without uid option show a 'nmt' owned filesystem ? Shouldn't it be root as well ?
I seem to have been misleaded by that one.
Doing some research on how usb drives were hot-mounted if found out that another recursive chown may well be launched on them too... (which could result in very lengthy plug-to-disk-ready time: see threads about this other issue).

Here are my findings :

When an usb disk is plugged into the system, udev daemon is requested to handle it.
/etc/udev/rules.d/syb.rules has a pattern matching for handling hot mounted partitions.
namely :

Code:

etc/udev/rules.d/syb.rules:ACTION=="add", BUS="usb", KERNEL=="sd[a-z]*", RUN+="/bin/usbhotplug add@%k"
etc/udev/rules.d/syb.rules:ACTION=="remove", KERNEL=="sd[a-z]*", RUN+="/bin/usbhotplug remove@%k"

this means that the binary /bin/usbhotplug is launched whenever a device matching sd[a-z]* is reported by the kernel.

usbhotplug is a mere script that simply echoes its argument to a previously created named pipe /tmp/devicepipe.

Code:

#> cat bin/usbhotplug
#!/bin/sh
/bin/echo $* >> /tmp/devicepipe

To find what script/binary use this pipe, I grepped its name and found :

Code:

>grep -r devicepipe bin etc sbin
bin/usbhotplug:/bin/echo $* >> /tmp/devicepipe
Fichier binaire bin/mounter concorde
Fichier binaire bin/fw_image concorde
etc/init.d/S01sysinit:mkfifo -m 0666 /tmp/devicepipe

Among the four candidates, mounter seem to be the one responsible for mounting (and unmounting) the hot-plugged devices. An intuition, don't ask me why.
mounter is launched as a background process at line 79 of S99misc-init startup script.

Summary: whenever a usb drive is hotplugged into the system, the kernel reports it to udev daemon which calls /bin/usbhotplug, which writes disk device name to the /tmp/devicepipe named pipe, which appear to be continuously read by the 'mounter' background process.

Now, looking more in depth into /bin/mounter we get :

Code:

>strings bin/mounter
   [... omitting many strings ...]
USB_DRIVE_%c%c%s
%s/USB_DRIVE_%c%c%s
/opt/sybhttpd/localhost.drives
devicepipe
read
add@
remove@
chown -R nmt:nmt %s > /dev/null 2>/dev/null

And here comes the long dreaded chown -R nmt:nmt %s > /dev/null 2>/dev/null...

My conclusion it that there actually are two bugs in the startup procedure, both are linked to this recursive chown.

• At each bootup, the internal hard drive is mounted as root and triggers a recursive chown.
• Whenever a usb hard drive (or usb stick) is inserted, its partitions are mounted as root and a recursive chown is performed on each of them (I must verify this behaviour). This actually is quite bad if you happen to plug a linux formatted drive where file permissions are important.
  

[laureck]

just catched the vilain!
I tried the following :
After a fresh boot I plugged my external usb drive to the pch then I repeatedly looked for a chown process and here it was :

Code:

24198 root       1936 S   sh -c chown -R nmt:nmt  /opt/sybhttpd/localhost.drives/USB_DRIVE_A-1
24199 root       2008 R   chown -R nmt:nmt  /opt/sybhttpd/localhost.drives/USB_DRIVE_A-1

After about 30 seconds, the process finishes and a symlink /USB_DRIVE_A-1 is created to the mounting point.

This mean that you should never plug a disk in the pch if it's used in another linux box => all files will be owned by user 1001 afterwards.

[laureck]

Updated post #1 with a summary of posts #1-#19 for thread clarity.

Also, I noticed that the pch was able to browse and read all the files of any mounted drives REGARDLESS of ownership nor permission: i.e. a media owned by root with unix permissions 000 inside a directory with permission 000 can be succesfully browsed to and displayed by the pch.

This means that the chown is quite useless when it comes to media playing. Its only use is that it allows files to be read/written through smb/ftp. Again, there's no way to create files with bad owner/permission on the pch by regular means so why bother with a blocking lengthy chown at each boot and each usb drive plugging ?

I do not own a C200 but it seems that it shows the very same bug.

I wish this problem could be at least acknowledged by officials

[McOrmick]

(11-09-2009 09:46 PM)laureck Wrote:  I wish this problem could be at least acknowledged by officials

Indeed ... with 3 1TB HDDs connected over USB this process takes a very long time.

Is there any news on the bug report? Would be good to see Werner or Steve acknowledge the issue or share with us the opinions of the tech guys.

[laureck]

(11-19-2009 09:28 AM)McOrmick Wrote:  Is there any news on the bug report?

Not much. To my knowledge, dc11ab relayed this bug to the private and low-noise-therefore-more-read moderator subforum about a week ago. No new news though.

[dc11ab]

Sorry to say, I haven't heard anything since. If I do I'll update this thread with any news.

Note that moderators do not have 'behind the scenes' access to info on priorities on bugs, architectural design changes (fundamental or not) etc.

[lordy]

The USB chown is nasty.
I dont think the internal HDD chown is too bad. Not brilliant, but the media files are big so there are not usually that many files.

The following is on a 1TB HD that is 97% full...

Code:

# date ; chown -R nmt:nmt /opt/sybhttpd/localhost.drives/HARD_DISK/ ; date
Wed Dec  2 11:31:23 GMT 2009
Wed Dec  2 11:32:37 GMT 2009

Code:

# df -k /opt/sybhttpd/localhost.drives/HARD_DISK/
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda3            960681352 927155376  33525976  97% /opt/sybhttpd/localhost.drives/HARD_DISK

Code:

# ls -Rl /opt/sybhttpd/localhost.drives/HARD_DISK/  | grep '^-' | wc -l
37478

From a workaround perspective, (Ie assuming a bug-fix is not happening anytime soon!...)
It looks like /etc/init_nmt is called from /etc/init.d/S99misc-init ?

[Edit removed wrong info - may fix soon!]

Meanwhile I think CSI should get a TweakNMT© screen.

[defer]

+1 for the TweakNMT© screen.

Because the tweaking by myself is "chinese" for me !

[laureck]

(12-02-2009 12:54 PM)lordy Wrote:  I dont think the internal HDD chown is too bad. Not brilliant, but the media files are big so there are not usually that many files.
(...)

Code:

# date ; chown -R nmt:nmt /opt/sybhttpd/localhost.drives/HARD_DISK/ ; date
Wed Dec  2 11:31:23 GMT 2009
Wed Dec  2 11:32:37 GMT 2009

I totally agree. 1'14" is totally okay for me (a little annoying for a hotplug time but totally okay for a boot time).

The issue is not correlated to the total size but to the number of them.

This means no problem for large video files but potentially much more (useless) waiting if you have large songs collections, photos collections or if you use the pch as a disk backup utility (which I do).

For me this adds up to more than half an hour...

(12-02-2009 12:54 PM)lordy Wrote:  From a workaround perspective, (Ie assuming a bug-fix is not happening anytime soon!...)
It looks like /etc/init_nmt is called from /etc/init.d/S99misc-init ?

Prior to calling /etc/init_nmt this script calls several scripts that reside on the HDD including...

/opt/sybhttpd/sybhttpd

There may be scope to do something with this (eg swap out /etc/init_nmt and /bin/mounter with modified versions )

Of course it would be better to fix it properly.

Aww, nice idea I didn't think about /dev/hda2 partition 'code injection'. Will investigate this.

Thanks for your participation, I was feeling quite alone lately.

[lordy]

Oops I had some wrong info in my previous post. There are no direct calls to a any program on the /mnt partition from /etc/init.d/S99misc-init
Used to be to btpd , but that's been commented out.

[ should have done "find . -type f | wc -l" in my last command. What was I thinking! ]

[werner]

hi all,

i forwarded the thread to our engineers and wait now for an answer for them, if i get something i will let you know.

Regards,
Werner

[laureck]

Thank you soooo much!

[oxonian]

I just had a quick search for this issue again as I needed to reboot my NMT device and came across this thread.

Each time I reboot I remember it,, In fact its one reason I keep the device on 24/7. While rebooting it table about 20 minutes. This is because I have all my Music, Videos, backups and project files on the device for networking backup and file storage. IF you've only got 1000 or so video files I can imagine this not being much bother.. But when I kill the box and am forced to reboot, then wait so long, its kinda annoying.

This thread seems to have stopped chatting, is there another thread, or people focusing on what Console to buy for Christmas.

Anyway, I've read most of this thread and can see ideas and hints and also a mention of managing to get some words to a Sigma tech support... Do they exist ? I was always wondering how I can report issues, bugs and suggestions direct to a real list instead of another forum that gets as read as my IT Support desk emails do

Also, a copy of the source files would be great, then I could modify it myself.

Anyway, My SOLUTION to this chmod issue is simple. Allow it configurable. That's it.. Some NMT Flashed flag, or even a file in the root to state the fact... if it detects /.leave_me_alone_you_evil_chmod then don't do any clever recursive modification to my HD.. IF the MNT doesn't work, I'm quite able to stick my HD into a Linux box and fix issues, or reboot after removing this file.. whichever.

But as others have mentioned the NMT device can play files regardless of their permissions, then I don't see the need for it.

It will also allow me to control my SSH, Samba user list easier, as their /home folder will work without me checking in my own scripts.

Andrew