Posts by dziekon

Quote from votdev

Start developing or contributing to the project does not mean you need to know how the whole system works. If there is a problem you can digg into it and try to fix it. To do that it is NOT necessary to code a single line, simply find out what is going wrong, check the service config files, try to manually fix it, test it and open a bug report with a detailed description what is going wrong and how to fix it. The code to finally fix that can be done by someone else with deeper knowledge.

It's clear that this kind of help is currently possible, not denying that. But there's really a lot of place for improvements, mostly in the part about actually fixing (or implementing) things. The problem in this project is with the so called "someones with deeper knowledge". There's not a lot of them, and that seems to be a big bottleneck. And it's not only about the base project itself, it also applies to the plugins, which as we know, need a solid interface coming from the base project. If the project is supposed to be extensible via plugins, but making plugins is not easy, then we end up in a situation like we have now, where plugins are barely kept alive. There's a lot going on with the proposed dockerized solutions, which I personally really like and I'm fine with, but even with that I still think there's a lot of place for plugins (even if they would be used as simplistic frontends for dockerized services).

Quote from votdev

But do you know how often this happens? ZERO times. No help from the community. Nothing. Only complaining that feature x or y is not working.

Well, in the next post you've pointed yourself to a community provided PR, so that's clearly not true. But I get what you are trying to say - that these sort of contributions are really rare, which after all this time must be a huge disappointment. However, don't forget that there's always the other side of the coin. Your point of view, your view on the code, its state and its documentation is from a perspective of the original author. It's obvious that for you it's all well (or good enough) written, everything is clear and shiny. But that's not necessarily the case from the point of view of an "outside" developer such as myself, a new commer who maybe wants to get to know the project better, try to improve it. However, when I saw the sad state of this community, I was really disappointed myself and simply backed out of any development in the plugins that I wanted to improve. And just before anyone accuses me of being an ungrateful bastard (looks like someone already said that, just not explicitly...), I really do appreciate all the fine work everyone has done so far, but my dev experience so far was not that great and simply made me rethink my further contributions and even further usage of OMV (because I'm not 100% sure if I should trust it to keep my data, if the development process is not stable itself; I know that a lot of people have only the positive opinions, which is great, but the developer inside me does not let me ignore the mentioned secondary flaws). OMV 5 and some of the rumors have brought back my hope, but I'll see how it goes...

But I digress, back to the topic. It's pointless to constantly reminding everyone that it's "just about reading the code", because it's not. This is a big project, with a lot of technologies involved (frontend, backend, OS-level configuration, that's quite a lot), with a quite complex, somewhat outdated architecture and old libraries. There's a lot of indirections going on, a lot of things people might simply not understand from the get go. And don't forget, that same as you, every outside contribution is also made in the spare time of any constributor, so it seems crucial to not waste that time on long analysis, and instead get to work as quickly as possible. And before anyone says that documenting things might also be a waste of time, just consider that there's also a chance that this "waste of time" might possible bring new developers which will in turn use their time to make up for the previous loss. It's not a guarantee, but in my opinion it's worth the risk.

It's definitely possible to track down all of the complexities and finally have the bigger picture, but it would be so much easier if it was already written down somewhere in a concise way. Then, once the initial "get to know each other" is done, the argument about "just reading the code" makes more sense, because one can then infer a lot by comparing it to something they've already seen before.

As with a regular job, it's usually good to introduce the new coworker by sitting with them and explaining the key concepts of the job they are supposed to do. This usually lets them to get up to speed more quickly, be more productive within a few days, rather than weeks. And isn't that what what you would like as well? More developers who quickly catch up and can do something in the code? This obviously is not just a "regular job", so it's harder to onboard new contributors, but it's definitely not impossible.

---

To sum up my thoughts - I get that what I just said is a ton of work and is not that simple, but doing nothing about it is the reason why OMV and more importantly, its ecosystem, is not as good as it once was. I guess this is still a fine product, not implying that it will fall apart or anything like that... But it's not very appealing to new developers, hence the lack of contributions and the stagnation of plugins. And no, I'm not writing this to pick up a fight or personally annoy anyone. I'm writing this from a perspective of the said new comment, trying to present my point of view.

I'm not saying that everyone will immediatelly stop being jerks about bug fixes, I'm not saying that there will be a surge of new developers joining the community, it all might be just a big ball of cliche ideas, but from my perspective it seemed like a lot of people involved live in denial about the situation. This might be an open source project, no guarantees, no obligations to support anything... but the existence of project's momentum reflecting on the community and the users cannot be denied, and should be taken into consideration.

Quote from ryecoaaron

I agree. I will be the first to say I don't have time to do more than I am doing. Do you have a solution or idea for any of these problems? I don't know how to fix the chicken and egg problem of we need more devs but we need more docs to get devs.

Obviously, that's a really hard problem to solve. First of all, it's best to acknowledge that this problem won't go away immediately, there won't be just one solution to it and it will take a lot of time and effort. Baby steps is probably the way to go, trying to solve one problem at a time.

I don't think it's a chicken and egg problem at all. There are a few developers already, people like yourself and Volker. Since the OMV 5 is now released and stable, it might be a good idea to dial down on bringing new features (unless they are crucial to the "community driven development" of course) and try to work on the documentation side of the project. For example, as Volker already mentioned, there's the RPC service which really resembles REST-like interfaces, but it's documentation (or rather, how to use and what "methods" are available) has terrible DX. New developers are forced to read through the services' code to find out what is available. It's not impossible to understand it, but it's definitely not making it easy for anyone other than the "old farts" that are already used to the old ways. Just because something exists, it doesn't mean that it's good.

But that was just an example of what could be done to keep more developers active. Here are some other ideas:

• Provide clear overview of the overall architecture of the project (development-wise, so not from the end-user's point of view). How certain things are achieved internally (eg. how configuration is applied, how can a RPC / REST endpoint be implemented, how to enable the newly implemented feature), how does the data flow in the system, if there are any caveats, workarounds that the authors are not happy about but still everyone should know about.
• Be clear about the project's status and roadmaps, what needs to be done or what is planned to be done. Bug fixes are not the only thing that people can contribute in their spare time. The mentioned REST interface, that's something that maybe a more experienced person could help with, provide some feedback (in RFC fashion or even as a part of code review for someone else). I've also heard rumors about some plans to rewrite the UI to Angular, so again, there might be someone who is experienced in Angular and would like to help with even just a tiny bit of the UI. Or maybe it would be better to use a different framework, like Vue, or React or whatever, maybe there would be more people willing to contribute in way X rather than way Y?
  Github should have all the necessary features to make things work, they just need to actually be written down and put in the right places.
• Improve the development experience itself. As far as I can see, the development guidelines are rather brief and don't give a good overview of what's it going to be like to fix a bug or develop a new feature. I've seen that OMV's repo contains some Vagrantfile which I assume could be used to quickly bootstrap a dev instance in a VM, that could be included in a guide (or at least mentioned that it exists). As far as I see, the new Salt-based config process is not mentioned in the contribution guide, I assume that this might simply be a fresh thing and there was no time to add it yet, but probably should be there as well. Maybe some sort of complete example of feature introduction, to simply show how things are usually done from start to finish? What I mean by all of that is - make the dev onboarding process as easy as possible, explain as much as you can. And most importantly, think twice before saying "something is doable, just read the code god dammit!" - if something about the development process is not clear, maybe it simply is... just not clear, and should be changed. Not everyone is a veteran of OMV, and any behavior that might be perceived as elitist certainly does not help.

Even though it resembles the mentioned chicken / egg problem, it should be said that at this point it will be much easier to improve the documentation to attract new developers, rather than to get new developers to improve the documentation. It's would be like trying to use a gun without having any ammo, best you could do is to attack someone by throwing your gun at them.

Quote from ryecoaaron

I know about them when they are released just like you.

To be clear, that was not a reference to you, just the overall situation.

Quote from ryecoaaron

Upgrades from 4 to 5 are not supported (not my decision). That is why there is no omv-release-upgrade or an official post on how to do it like previous versions. I came up with a process that works ok but I'm not making it a front and center post because I would rather people not do it.

I know that. What I was trying to say that even though it's unofficial, it already lives on its own - people link to it all the time. Even if it's buggy (not saying that it is), it might be less harmful to clearly state that in some kind of OMV's FAQ, rather than keep it unleashed like it is now. But that's just a thought.

Quote from ryecoaaron

I will admit I have little patience for nitpicking. If I am preventing the project from growing by my actions, I would gladly hand over the work to someone else.

Is it really that bad? Every solution seems to point to "spend more time and fix everything".

I'm not here to change anyone character or attitude (and btw, this is not a note about you personally, just a general one; I actually don't know who exactly was involved in the "fight"). I understand how annoying it is to hear "fix it for me" all the time, but that's simply how open source projects usually are - most people want to use it, for it to work properly, give nothing in return and complain when something does not work. But in some rare occasions, people have legitimate concerns, and releasing frustration on them like on every other idiot only makes the overall community look bad. The users... usually don't care. But the developers do, since they are the ones who might be dealing with certain people along the way. Actually, even the users are important in this picture - they are the QA after all, right?

And again... I'm not saying that anyone should immediately become the next saint or whatever. Just... be aware that your actions have consequences, and even though you might be invaluable to the development process, it doesn't automatically absolve you from being a jerk or whatever (again, if it's not clear, I'm not talking about you specifically, but rather... in general. I had a feeling earlier that you've taken some points personally, that was not my intention, sorry). And no, by removing that kind of person from the development process (whether it's voluntary or not) is not a solution. Like with a certain drama here on this forum, it only shows that this kind of behaviour is childish, and nothing else.

Quote from ryecoaaron

I'm in the same boat about 99% percent of the time. But people can develop against the stable version. You don't have to develop against what's next.

As mentioned earlier, this is more about knowing the roadmap, the next ideas being implemented, and being able to see their status and maybe help with their development. Volker does not have to implement it himself, it may take a lot of time but maybe, just maybe, someone will finally show up and help with more and more things.

And don't forget about the bus factor, which is currently virtually equal to 1. To me, it's even unclear if someone else would be able to re-create the .iso if Volker would leave the project.

Quote from votdev

And don't tell me that it is not possible to get familar with OMV without huge docs.

This first sentence already tells me that you completely missed my point about growing the community around this project. I see that you've already jumped to wrong conclusions. Instead of getting mad at me and immediatelly attacking me, take a step back, calm down, maybe read my posts once again.

Quote from ryecoaaron

What do you mean by "development environment"?

It would be nice if people would help develop the development environment. I don't know who else is going to create it. Volker maintains OMV and I maintain plugins. This is not enough people.

The problem is that OMV doesn't have enough documentation nor its code base is easy to read and understand. In addition to that, anyone who wants to do anything is simply left on their own, because "you're supposed to learn the ways around like the rest of the old folks did". While I understand that it's mostly a time problem, everyone is doing this in their own free time, but using it as an excuse won't lead to anything other than stagnation. You can write as many explanations and excuses as you want, but it doesn't matter - it's just a fact, and the number of recent contributions from people other than the longtimers is the best argument supporting that. What this project needs is an easy way for developers to get to know project's internals, its mechanisms and flows. OMV is pretty complicated, so the easier this onboarding process is, the more developers will be willing to help.

There's also a big problem of no clear path about release management, future plans, how bugs and tasks are managed, etc. Maybe if you dig deep enough into this forum, you'll be lucky to find anything meaningful by accident. For example, the migration process from v4 to v5, even though it's not official, should have been extracted into a separate, highly visible article a long time ago. Instead, people still ask about it and are still being redirected there once someone is kind enough to share a link.
Not to mention that people pointing out certain problems are being left with dubious and unfriendly answers (like the recent banter related to OMV 5's release notes or something similar to that). And don't get me wrong, I don't mean that everyone should love each other and barf rainbows, I'm simply saying that snarky answers to legitimate questions will drive people away. Again, you can say all you want about this project being available for free, that authors are not responsible for anything and so on as so forth, but at the end of the day, it's just a waste of everybody's time - people will get mad, you'll get mad and nothing will get fixed or developed. I'm pretty sure that some people will see this post in the same way.

As for the mentioned REST API. I'm sure it won't be the "next big thing" in the plugins development process, but I believe that it's existence and the related refactor might be really beneficial to the codebase. It might clear it up a bit and make certain things much simpler than they are now. For example, the ZFS plugin uses certain built-in methods of the OMV itself. Having a REST API with good enough documentation, or even a well-documented service related to a specific endpoint, could help to make certain things easier for plugin developers. In addition to that, any UI-only plugin could potentially be developed in separation from the backend's code; or maybe some statistics-related plugins could consume these API endpoints on their own, living completely outside of the OMV's lifecycle as stand-alone services based on docker... who knows what fantastic ideas are being held back by the lack of a legitimate and easy to use interface.

However, that's not why I brought it up. I've mentioned it as an example of an initiative being developed "in the shadows" - there's no way of knowing what is its current development status, what's the plan, what's left to do or even how can anyone help with developing it.

Quote from ryecoaaron

100+ employees helps.

Having a healthy and well maintained development environment helps as well. OMV doesn't have that, so no new developers want to join this community as developers or plugin creators. The "proposed" REST API could help with this a bit, but there's no way of telling whether this idea is being developed or not at all.

Quote from MarcS

The Snapshots/Clones were indeed created by Docker but I dont understand why so many (I counted 15 Snapshots although I only have 2 Containers.)

This part of the ZFS Driver guide document explains said behaviour:
https://docs.docker.com/storag…mage-layering-and-sharing

The plugin's code is hosted on Github here:
https://github.com/OpenMediaVa…lopers/openmediavault-zfs

However, you should be aware that the current master branch contains changed that were not released as a deb package. You can always look through commits history to find where the version was bumped and treat is as the "latest released version".

Quote from ryecoaaron

This is a huge change. I didn't remember it being merged. I need to look at the changes (again?). I will try to test it but I don't normally use zfs. So, I'm not the best tester. I guess I can put it in the testing repo.

The changes were tested and merged (but not deployed) on Oct 23, 2018 by subzero. Putting this in the test repo for wider audience might be a good idea, at least as long as someone is willing to test it.
However, I do not promise that the PR I have in mind here solves the same problem as stated above. It sure does look similar, but who knows what exactly happened.

Quote from getName()

Ah right. If they got a partition table or are marked as part of lvm, zfs does not allow to use them to assemble a raid. This is general zfs behaviour though.

This is not actually ZFS's fault, but rather how OMV's raid management service lists potential candidates - potential RAID candidates have to be completely "clean" of all partition headers to be considered "usable". Plus, it has to be a "device", not a "partition" (still, this is not a problem for ZFS itself, although probably not recommended). ZFS's plugin is using this service's methods to enumerate all drives that can be used in pools, therefore its behaviour depends on how OMV treats such drives. So yes, the best bet in this kind of situation is to completely wipe the drives.

However, I do recall there was a problem with some drives not being properly "cleaned" of all labels after pool destruction (something to do with paths to the drives being incorrectly enumerated by the plugin). I suspect this is the cause why @devnet was not able to see the drives on the list. The issue has been fixed, but I believe this fix has not been deployed as as new version of the plugin yet.

Ok, so at least we know it's not ZFS's fault.

Are you 100% sure that you've wiped these drives correctly? That means wiping the data, filesystems & partitions as well?
Using "WIPE" option in "Storage > Disks" section would do just that, but there's always a chance that you did it by hand and something is still residing on these drives.

@skylamon can you check if you can see any devices in "RAID Management > Create" form? ZFS plugin is using RaidMgmt Service's "getCandidates" function to create the available devices' list, so it looks like it might be OMV's fault, not plugin's.

Linux 4.18.0-0.bpo.1-amd64 kernel is known to be working FINE with ZFS, btw.

Quote from gderf

I checked the dates on the two files involved and they are dated yesterday so I know the 4.1 update took. And a year or so ago I patched those same two files with the same commits. Neither fix worked.

Best way to debug this problem would be to see all styles applied to that list using DevTools, and then compare it with someone else's styles (who does not have the same problem). This way could at least provide an indication of the root cause of the problem.

Quote from ryecoaaron

While it would make sense to lock it, most people would complain since docker updates often. most updates don't break the plugin (first in at least a year). What really needs to be done is someone needs to be maintaining the plugin and testing against a pre-release docker repo.

I assume you're talking about minor updates, since 2018.09 seems like a major one (previous major was three or four months ago it seems). It should be possible to lock dep to major version, while still allowing minor patches.

In general, I understand that sentiment, but it's not good to have production systems breaking because it's "unlikely that the new version will have breaking changes". We're talking NASes here, where people expect stability to be a first class citizen of the project, so stability should come first, especially when maintainers do not have time to accommodate for breaking changes. It's better to have an outdated set of features, rather than no features at all. What's more, this approach gives you, the maintainer of the plugin, enough time to accommodate to any changes in your free time, on your own terms.

I won't push this discussion any further, and obviously I have no way to force you into this approach, so you'll do as you think is the best. But still... please reconsider my proposal for the sake of project's stability.
BTW, I'm not trying to distract you from fixing the problem, just trying to prevent same "disaster" in the future while it's still time to take notes from this accident.

Quote from ryecoaaron

It isn't locked to that version. It requires that 18.09 or newer.docker-ce (>= 18.09)

That's what I said. I've proposed to LOCK IT to that version, not the other way around.

@ryecoaaron I just saw your commit on omv-docker-gui's repo. Shouldn't you lock docker-ce dependency to 2018.09 instead of happily allowing newer major versions? This should prevent problems like this in the future, where an unknown update of a dependency breaks many users OMV configurations. Obviously, this will lock users to the "last known-to-be-working version", but this seems like a better approach than "be up to date and potentially break things".

Quote from Palulukas

Hello everyone and thank your very much for your answers.

I looked up some boards from asrock and asus. From what I can tell only the itx boads from asrock and the prime x370 pro from asus do support ecc correctly.
itx is not what i look for but the prime x370 pro looks promising.
Does anyone have another advice for an compatible motherboard?

In case, I choose the AMD path: I need a graphics card for the initial install and after that I can pull it out again, right? Do modern (consumer grade) motherboards support operation without a graphics card?

Kind regards

Palulukas

Display More

You should look harder then I guess. For example, asrock's X370 taichi (which is an ATX board) supports ecc on both summit ridge and pinnacle ridge according to their official specification. I'm pretty sure I heard some info about Wendel from L1tech running 1st gen ryzen with ecc memory on gigabyte boards as well.

As for the GPU, I think this is usually the case nowadays. I'm running X370 itx board from ASRock, besides the beep on startup (which indicates a problem with GPU, but does not stop the POST) it works fine without a card. Obviously, your mileage may vary, so you should do your own research over the Internet once you pick your board.

Quote from TheLostSwede

As per ASRock's website "For Ryzen Series CPUs (Raven Ridge), ECC is only supported with PRO CPUs." So that Ryzen 3 1200 doesn't support ECC on their boards, or anyone else as far as I'm aware, hence my post above.

Raven Ridge are Ryzen APUs, not all Ryzen CPUs (through, I admit this notation is very confusing... shame on you AsRock).

Maintaining "releases" section (or tags directly in the repo) has a benefit of easier "release point" discovery for developers. Adding a tag adds an easily discoverable point in repo's history which determines the boundary of said version, which for example can be used to produce code diffs or commit logs between versions. Really useful when someone is trying to understand why, when and how certain things have been implemented or changed (which cannot be comprehensively expressed through debian's package changelog; but it serves a different purpose anyway), especially in open source projects, where literally anyone can decide that they want to help move things forward (but to do this, they need to understand how people before them "did things"). Plus, it's like two or three clicks to do that on Github, so it's really not that time consuming.

Ok, so now, with my PR merged into master, I have a question for you @ryecoaaron (I actually tried to ask that question on Discord, but it seems you are not present there, and it looks like @subzero79 does not show up there very often) - when this is going to be released as a plugin update?
Or, to better phrase my question, what is the general plugin deployment cycle for all stuff released as a part of omv-extras repo?

Looks like my previous PR (#53 on Github) has not been deployed either, so I'm just curious what is the usual ETA of plugin updates. Obviously, I can (and already did) build it myself, but it still feels a bit wrong to use unofficial versions. BTW, what's up with "Releases" section of ZFS plugin? The last version there is marked as 3.X.Y, which is not in sync with the real state of omv-extras repo, and it doesn't look very promising for all new developers willing to work on this plugin.

Quote from ryecoaaron

There is no official notice and Volker hasn't said the core plugins will be going this route. I have said it many times since the downloader plugins weren't ported to OMV 4 let alone OMV 5. I am trying to get people to stop using the plex plugin. Here is my opinion on OMV 5 plugins (in list form):
...
If not on this list, it is most likely docker.

Ok, even though raulfg3's post sounded alarming, this is exactly what I expected ("low-level" stuff still as plugins, and "complete solutions" wrapped in Docker containers to lower the maintenance burden). Thanks for clarification.