missing file issues - RootFolderBrowser.js

  • OK, ryecoaaron this topic seems more or less ontopic. To go on from the asual mantion of Extras which seem to trigger you.

    https://github.com/openmediava…92#issuecomment-809518469


    Since i can not reply to you in place and it would be offtopic anyway, let me clear it here.


    Quote

    Error #0:

    OMV\Exception: Failed to open include file 'js/omvextras/window/RootFolderBrowser.js'. in /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc:42


    > The webui should not stop working after omv-extras is installed.


    I agree, but it does, not every time but enough to make it common.

    When the whole of UI of OMV stops working - this is hardly to perceive differently than "system is falling apart".

    Now there can be "not my problem" game between OMV core, OMV extras core and specific plugins.


    How ever from user perspective it looks like that.

    1. User installs OMV - and it works

    2. User installs OMV-Extra and some Extra-provided plugins, and then his OMV box is bricked.


    You may be triggered by using "bricked" word, but then we have to remember


    > Running a GUI on OMV is absolutely not supported because OMV and GUI's work against each other. OMV is the master of the system by design.

    https://github.com/openmediava…91#issuecomment-808946830


    And there is no other UI than WebUI. (for example Mikrotik routers have dsktop applications as alternative first citizen UI).


    So, if WebUI is not available, then system is bricked, from user perspective.

    What users sees is that he enters OMV UI, checks some checkmarks - and OMV UI is not available any more.


    https://en.wikipedia.org/wiki/Fear,_uncertainty,_and_doubt


    > Even if it isn't installed and a plugin needs, it doesn't break the web interface. It only breaks the folder browser.


    The missing file names imly this, yes. Howeve reality is different, WebUI stops to work.

    I can speculate that WebUI tryes to force me to some "panel open in previous session" and that panel happens to have browser. Or anything.

    However, on occasion, mere installing Extras (core and some plugins) makes fresh one day old OMV box bricked.


    Voker sais i have some sad luck to step into bugs, and that is somewhat true. However not me alone. One of top lines in google leads to Login Error and Reappearing Deleted Log Files


    The same story. Install something frm extras - get the whole WebUI bricked. Reboots do not fix it.

    It is not like page renders but some "directory listing" iframe does not appear. It that user can not login into OMV at all.


    I may say, there should be something fragile in OMV itself then, if lack of one file can break the whole UI, not some localized part of it.


    > It is needed by a handful of plugins that should install it just like any other dependency when you install the plugin. If that isn't happening, something is wrong with apt on the system.


    I do not think there is "something wrong with apt", would it be so, to borrow your words "APT would have thousands of people with issues otherwise".


    > omvextras-common is not needed by most plugins or omv-extras itself. It is needed by a handful of plugins that should install it just like any other dependency when you install the plugin.


    Now that is probably correct. But there is nothing in OMV Extrast the project that ensures it. Users keep stambleupon it again and again and OMV Extras the project says "have no idea what is deleting the files".


    You are triggered by "falling apart" but put on users shoes. You enter standard system UI, you click several "omv extras" packages, you click apply - and the sysatem is bricked and OMV Extras say "we have no idea". It doesn't sound like "fallign apart", not at all?


    > If something is removing that package


    I dop not believe it. it was a fresh one day old OMV installation.

    However would something really be deleting it than APT would delete also al lthe dependent packages, won't it?


    So it is most probably a long-standing problems with dependencies amonf OMV Extras packages, plugins including.


    > It would be interesting to have more details.


    Sure. But sadly there is no. I was battling with a totally unknown to me system, and other issues. So i can not remember much of specifics. I have a photo showing apt-geet window in process, but i am not sure iot was related to this or something else.


    I have no repro and little details i remember with certainty.


    > If you can't close the window, you can alway refresh the page


    Eventually i did. Maybe that was when the whole WebUI stopped working, or maybe not, i am not sure.


    User can also open their browser's DevTools and enable the close button or remove the frame/div altogether.


    But see, it is last resort. User has to be sure it is time to close the window.

    It would be disrupting to destory that frame while apt-get session is stil lrunning and the output is not finalized, won't it?


    But user has no way to know the underlying process is already finished and hardkilling the window is safe now

  • To go on from the asual mantion of Extras which seem to trigger you.


    I read all of the omv issues. You said omv-extras was falling apart and I wanted to know more.

    Which plugin? If I can't recreate this on 10+ systems and I don't get useful output, I don't know what to fix. The other thing that is very hard to debug is when people have many plugins that alter browser behavior. I have no plugins installed other than what the browser comes with. I have seen many times that plugins cause problems. omv-extras is a very simple plugin code-wise. I don't know how it could brick your browser (it isn't bricking the system). And most OMV users figure out ssh is still available and don't need a desktop environment. omv-firstaid is easy to use from the command line.


    However, on occasion, mere installing Extras (core and some plugins) makes fresh one day old OMV box bricked.

    I have literally done this hundreds of times in the last few months and don't have this issue. Since it is the browser having issues (extjs is client side), do you think it is possible the browser is causing this? The other issue is that extjs has not been changed in a long time. So, as browsers change, it is possible something might break.


    The same story. Install something frm extras - get the whole WebUI bricked. Reboots do not fix it.

    It is not like page renders but some "directory listing" iframe does not appear. It that user can not login into OMV at all.


    I may say, there should be something fragile in OMV itself then, if lack of one file can break the whole UI, not some localized part of it.

    Do you see thousands of posts about this on the forum? No. Then rare issue are hard to fix when not reproducible. Give me a list of steps that can reproduce an issue. If it happens repeatedly on your new system, it should be easy. Also, details about your system would be helpful - what media is OMV installed on, what cpu, etc.


    I do not think there is "something wrong with apt", would it be so, to borrow your words "APT would have thousands of people with issues otherwise".

    apt on *your* system not apt in general. I love apt/dpkg and it works well for me on hundreds of systems every day at work and at home.


    Now that is probably correct. But there is nothing in OMV Extrast the project that ensures it. Users keep stambleupon it again and again and OMV Extras the project says "have no idea what is deleting the files".

    Yes, there is something that ensures. The control file in each package determines if it is needed for the plugin and apt will install it if it is needed. See the exact list below:

    omv 5.6.13 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • > Which plugin? If I can't recreate this on 10+ systems and I don't get useful output, I don't know what to fix.


    Good question... And even if you could - it would only fix it for one specific plugin.

    I wonder if there can be some testing sandbox or something to make repetition of it impossible.


    That said, Valker said OMV6 is dropping all the WebUI and redoing it almost from scratch. So, maybe this issue would have less impact when OMV6 released (in two months? my, my...)


    > Since it is the browser having issues (extjs is client side)


    Are you sure? There is nothign new in server-side JS (nodejs as obvious example) and tome that clearly looked as server-side issue.


    Also, it was fixed by googling for the package name, logging into the box by physical keyboard and apt-get install.


    There was nothinf i changed in browser to start this problem, nor to fix it. It started after some OMVExtras were installed, it ended with apt-get install.


    > The other thing that is very hard to debug is when people have many plugins that alter browser behavior.


    True, but irrelevant here. The WebUI generation was failing on server side. Or at least with a file required but server-delivered UI program but not delivered by server. Whatever my brower would do or misdo - it was not relevant here.


    > The control file in each package determines if it is needed for the plugin


    That is not what i mean. What if plugin author would use the file but would not update the control file?


    You see, using any module from "dirty" system and assuming it is always there is easy.

    Without "clean system" test of every plugin it is very hard if possible to detect.
    And UI tests are hard to automate, though browsers today provide some API to make robots pretend human behavior.

    And making clean system tests of every combination of plugins would really combinatorial explosion.

  • the_Arioch

    Changed the title of the thread from “missing file issues” to “missing file issues - RootFolderBrowser.js”.
  • So the specific error is, again,


    Quote

    Error #0:

    OMV\Exception: Failed to open include file 'js/omvextras/window/RootFolderBrowser.js'. in /usr/share/php/openmediavault/controlpanel/controlpanelabstract.inc:42

    That is NOT happening in browser. That is happening in PHP and my client box has no PHP installed, it can not be the code running in the browser. Thus it happens in server.


    Now, i SSHed into OMV and i cleared cache in my client browser and WebUI still works.


    I downloaded NirSoft's ChromeCacheView and what i see in my client's cache is

    Quote

    What should all this mean?


    My bet is that:


    1. Yes, there is some client-side use for that JS library, and if it is missed, then SOME functionality ofd WebUI breaks.


    But not all of it. Never a catastrophic crash.


    2. Server side ALSO doies use this library on some occasion. When it happens - you have a catastrophic crash with GUI regardless of browsers.


    3. Server side PHP probably does cache the file. So once it is there in server-side cache the presense of APT package does not cause any immediate input. That is what creates the illusion "OMV works fine without that package, so it should not be on dependencies list" and reference to it gets trimmed from .DEB files

  • That said, Valker said OMV6 is dropping all the WebUI and redoing it almost from scratch. So, maybe this issue would have less impact when OMV6 released (in two months? my, my...)

    omv6 is not 2 months away. I haven't even started porting the plugins yet either.


    Are you sure? There is nothign new in server-side JS (nodejs as obvious example) and tome that clearly looked as server-side issue.


    Also, it was fixed by googling for the package name, logging into the box by physical keyboard and apt-get install.

    I'm referring to the browser lockup not the missing files. And there is no nodejs in omv. If you think omv is locked, login via ssh and check to see if engined is running - systemctl status openmediavault-engined or if omv-engined is using lots of cpu in top. If it says it is running and not using lots of cpu, the system is not bricked = client side issue.


    True, but irrelevant here. The WebUI generation was failing on server side. Or at least with a file required but server-delivered UI program but not delivered by server. Whatever my brower would do or misdo - it was not relevant here.

    You are combining the two issues. rootfolderbrowser being missing should not break extjs. I have purposely deleted the file trying (actually the entire /var/www/openmediavault/js/omvextras directory) to recreate and it doesn't "brick". I see an error in the browser debugger but everything keeps working. And I have all of the omv-extras instal led on multiple system.


    What if plugin author would use the file but would not update the control file?

    I am the plugin author. Who else do you think works on these plugins?


    You see, using any module from "dirty" system and assuming it is always there is easy.

    Do you know how the control file works? It creates the dependency on the omvextras-common package and installs it. It will not install the plugin without it.


    Without "clean system" test of every plugin it is very hard if possible to detect.

    I build new systems or revert to snapshots at least a few times a week. Using vagrant to build a fresh system is also easy to do.

    And UI tests are hard to automate, though browsers today provide some API to make robots pretend human behavior.

    Not doing that. Feel free to write that.


    And making clean system tests of every combination of plugins would really combinatorial explosion.

    You must confusing this project with a commercial project with lots of paid people working on it who can devote lots of time. I do what I can. if people find bug and I can fix them, I do.

    omv 5.6.13 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • That is NOT happening in browser. That is happening in PHP and my client box has no PHP installed, it can not be the code running in the browser. Thus it happens in server.

    Please pretend like I know how OMV works. That error is caused by OMV trying to cache the javascript. It does NOT break omv-engined (which is the php part).


    Yes, there is some client-side use for that JS library, and if it is missed, then SOME functionality ofd WebUI breaks.

    Right. Anything that needs to display a root folder tree when you click the folder button on some text fields. It doesn't do anything unless you click on it. Hence why it shouldn't cause a brick.


    Server side ALSO doies use this library on some occasion. When it happens - you have a catastrophic crash with GUI regardless of browsers.

    Javascript is client side (there is no nodejs on OMV). It is only cached by the server. The server is not using it.


    Server side PHP probably does cache the file. So once it is there in server-side cache the presense of APT package does not cause any immediate input. That is what creates the illusion "OMV works fine without that package, so it should not be on dependencies list" and reference to it gets trimmed from .DEB files

    This is just plain false. Please study .deb architecture more.


    I'm done discussing this. If you can give me exact steps on a fresh system to reproduce, I will try to fix.

    omv 5.6.13 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I suspected so, but are citizens allowed deviate from Party Line and question official roadmaps?...


    > If it says it is running and not using lots of cpu, the system is not bricked = client side issue.


    If the engine is notminally running but does not serve on HTTP anything usable but onle error message - then for user it is essentially bricked.

    If there is no "button to press" in intended UI (web browser) then it is dead.

    Surely, sometimes unbricking a fault device is not hard. And luckily it was not hard here. But still...


    OMV distro foes not even have apt-file installed. Luaghabl to you, perhaps, but for me Windows guy it was one of blocks.

    Last time i used apt was like dozen years ago and on apt/rpm distro, so i was just lost in the wood.


    > And there is no nodejs in omv


    I did not say there was.

    But i said the error message clearly shown PHP server-side code was trying to include JS file for whatever means.

    And when it failed - the whole WebUI failed.


    Again, WebUI engine was not locked, frozen or something. It was generating and serbing HTTP page, but that page was nothing but PHP error message...


    > It creates the dependency on the omvextras-common package and installs it. It will not install the plugin without it.


    See, we here are having an hypothesis, that


    1) there is some omv plugin (that is, DEB package in OMV's apt repo)

    2) that is dependent upon that JS file

    3) but that fails to include omvextras-common in its dependencies list

    4) so when it was installed it did not pull the "common" package and proken PHP pages generator.


    Do you agree? The rest was going from that hypothesis.


    Other than htat where an we find black cat in dark room?


    > I am the plugin author. Who else do you think works on these plugins?


    That is going personal. I was discussing generic issue, which can happen with any programmer making a plugin. IT should not matter if it was you or any other programmer, bugs still happen.


    > using vagrant to build a fresh system is also easy to do.


    ...for those who already doing it "few times a week" for sure.


    Well, you have some point. I can make VirtualBox and try to isntall fresh OMV like i did... But reproducing this seems only possible by chance.
    Unless OMV keeps the log of all user's changes done last two days, so i could replay my actions verbatim. Is there such a full log? Probably not.


    > You must confusing this project with a commercial project


    You seem to misread, i specifically mentioned combinatorial explosion to show how hard that is. to do.

    Stop being agressively defensive please. Right this moment "I do what I can" to find a shreudenbug in a large project i first saw day ago. And we can easily start blaming fest and "who does more", but it will not bring us anywhere.


    In the end it will be between finding methods to make it impossible to repeat - and keeping " I have helped many people on the forum with this" timeand again. And only those users who bothered to go to forum not giving up on OMV.

  • > Hence why it shouldn't cause a brick.


    But it did. I do not recall if there even was logon box or not.


    So there is something you do not know about OMV as whole. Otherwise you would fix it but it is reoccuring.


    > Please study .deb architecture more.


    Here is nothing special about .devb or .rpm or .tar.gz or even .msi


    There is formal declaration of required packages.

    And there is the content, the programs that actually does something.


    If program from package B needs to use file from package A - and package A was not installed - then the program fails.


    It is not .DEB-specific.

  • that is dependent upon that JS file

    3) but that fails to include omvextras-common in its dependencies list

    4) so when it was installed it did not pull the "common" package and proken PHP pages generator.

    If the file isn't installed because the plugin doesn't have omvextras-common listed as a dependency, then the php code will not try to load it and the system will not be broken. Post your /var/log/apt/history.log and /var/log/apt/term.log and output of: dpkg -l | grep open

    omv 5.6.13 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • I hate working with two keyboards, always keep keying into wrong box...

    In theory i can run xserve on Windows, and maybe i would even learn to do it eventualy...


    I am now reading through PHP sources... Can you brief me on how that &$inclist is seeded?


    I am really buffled that the fatal exception is thrown for "mere caching".

    From common sense, cold cache is only worth warning not outright failure.
    It feels there is more to that than caching.


    For what i see, every processed file can add to incList if it contains `require(` text.

    And `require` does not seem to be "caching" it more looks like sanity check.


    The PHP pages generator would be outright killed if JS page can not be found.

    If you are correct that JS is never executed server side, then it is overachieving.

    Perhaps scanFiles should had an extra parameter, so it would only give WARNINGS not hard crashes if JS file missed.


    Another idea would be to make incList containing tuples, not values.

    Both names the files that IS required (as it is now) plus the source of requirement.


    If so then the said error would look like that

    Quote

    OMV\Exception: Failed to open include file 'js/omvextras/window/RootFolderBrowser.js'.

    This file is required by ''some-OMV-component'-which-is-probably-at-fault"

    So the first next time this happen - you know where to look for culprit.


    Those functions are marked as protected not private, so frankly i fear to patch them in place... But was it my program i would probably modify them so, rather than keeping answering the same questions again and again. Debug information is always handy...


    ---


    w.r.t. "dpkg -l" i'll d it, but problem is - i can NOT reproduce it. Despite removing the package.


    Was it some fleeting race condition? Or anything else...


    Frankly, 1 pm in Moscow and probably 3 pm in Germany. Perhaps we should call it a night for now. MY family is already angree for me "toying with computers" :-D

  • There also is eipp.log.xz but forum prohibits attachign it, perhaps it is less needed

  • After looking at your output, you shouldn't have been able to remove omvextras-common without removing openmediavault-mergerfsfolders. When I looked at the control file, I kept mistaking omvextras-unionbackend for omvextras-common because I was only looking at the beginning. If fixed this in mergerfsfolders 5.1.2 and it is in the repo. I guess most people aren't using mergerfsfolders or they install one of the other plugins that did have the package in the control file.


    And just a note... I broke this on March 18th. So, it hasn't been broken long.

    omv 5.6.13 usul | 64 bit | 5.11 proxmox kernel | omvextrasorg 5.6.2 | kvm plugin 5.1.6
    omv-extras.org plugins source code and issue tracker - github


    Please read this before posting a question.
    Please don't PM for support... Too many PMs!

  • Code
    grep -rn /var/www -e 'RootFolder'
    /var/www/openmediavault/js/omv/module/admin/storage/mergerfsfolders/Folder.js:22:// require("js/omvextras/window/RootFolderBrowser.js")
    /var/www/openmediavault/js/omv/module/admin/storage/mergerfsfolders/Folder.js:31: 'OmvExtras.window.RootFolderBrowser'
    /var/www/openmediavault/js/omv/module/admin/storage/mergerfsfolders/Folder.js:71: Ext.create("OmvExtras.window.RootFolderBrowser", {

    Am i right that "OmvExtras.window.RootFolderBrowser" and "omvextras/window/RootFolderBrowser.js" is representing the same data?


    If so, it would not be hard to make dependencies list automatically, by parsing JS files that have EXT's REQUIRE arrays and updatign control files.

  • Ddouble shot then. You made an answer while i was formatting my own.


    Now the general questions remains....


    1. WHY the OMV WebUI was breaking after installing mergerfs but is not breaking now?


    Is it that PHP requirements checker only runs after new packages added not removed? Or becase i removed the package from SSH and apt-get not from WebUI itself ?


    That check routine clearly run before but is not running now. This feels indeterminate, random, Shroedinbergish.


    I restarted the engine, and it did not detect file being missed. This seems to open the way for misconfigurations.


    2. won't it be good to have script parsing JS and PHP files and auto-updating dependencies lists?

    And make this requied step before publishing. Would preclude such class of errors from happening.

    Testing UI by piloting browsers is hard. But this seems to be doable.


    3. hard crash server side is uncalled for. You (OMV project as whole) do brick the system that otherwise would be somewhat useable.

    It is like, "never remove last admin user from OS" and other "don;t shot yourself in the foot" thumb rules.


    When system reports errors but does not fail to work - user can at least do something. Restore it from backup, uninstall recently installed plugins, reinstall recently removed ones, something! Core UI functionality should ntot totally fail to work just because purism.


    Reporting missed files is good idea. Bricking the box and letting users outside closed door - is not.

    You personally did not believe this could be happening and repeated time and again this is client-side problem not server-side.

    Because... well... this is jsut not the way how remote servers should act, agree?


    "fail early" is about specific funciton calls, not about the whole system.

    WebUI should not deny 100% of its functions when 1% of its functions is failing. It is not reasonable at all!


    4. my grepping seems to show that only two files are using this ScanDir infrastructure


    controlpanelabstract.inc and administration.inc


    and it ONLY calls them from getJavascriptIncludes function.


    let's refactor them to have tuples in the incList and to keep tracking WHAT was causing the requirement.


    ----------


    one more question, is apt-file broken in debian ?

    apt-file list omvextras-common returns nothing at all!

    Man apt-file claims it should show the content of package even if it was not installed.

    And the apt/rpm distro i used decade ago did....

    Amm i missing some options i should provide to apt-file or it is cannot do today?

  • > WHY the OMV WebUI was breaking after installing mergerfs but is not breaking now?


    Managed to reproduce it by finding and killing cache file...

    Still not totally hardening it.


    Now see this:


    This would have saved you and me few hours of debugging and arguing about ghosts, won't it?

  • Yet better like this.



    Now user is alerted about very bad condition.

    And alert is made in "cave age" style totally independewnd on any JS library (which we fear might be badly broken already and non-funcitonal at all)


    At the same time, it does not preclude user from making basic changes in OMV box.

    The error is detected, displayed, but the OMV box is NOT bricked.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!