Integrated VEL & JED, Please

integrate.md

Integrate the VEL and the JED

Introduction

I strongly believe the JED and the VEL functions should be more tightly integrated. The VEL's usability and visibility would escalate incredibly with such a change, and adding the VEL as a "feature" OF the JED would allow both teams to have a greater positive impact on the joomla community as a whole.

Reasonings

Record De-Duplication

Currently "vulnerability" information for extensions is not maintained where that extension is most prominently accessed. Instead vulnerability information is stored on the VEL, in a static like format with no connection to the JED listing.

Appending VEL information to a JED listing would mean that the extension has only one record within the Joomla.org family sites, and users would be able to review that extension’s past and current vulnerabilities within the context of the JED, where they most likely found the extension in the first place.

Increased Usefulness To The Community

The VEL property is less functional than the JED. Searching, filtering, and ordering are all features that the JED has implemented well. Any record searching utility, like the VEL portrays itself to be, should have these features.

Monitoring of non-JED extensions

One major reason that the VEL is not part of the JED is because the VEL is able to then “track” non-JED distributed extensions. This is counter productive to the way Joomla has positioned itself to developers.

The community of Joomla decided many years ago to support developers who play by the community’s rules. The VEL is doing a disservice to very intentional decisions the community has made to support our community by tracking non-JED extensions. Joomla.org property sites should not be inconsistent.

Access of VEL information via JED API

The Joomla Install from Web feature, although controversial, is a huge move forward for our community. Yet that feature is less useful, and detrimental to the image and brand of Joomla if it has poorly maintained, but one-click-install accessible extensions on it. Having an extension’s VEL history log within the record would increase usefulness and functionality to install from web users considerably.

Better Extension Developer Accountability

Because the VEL has relatively low visibility in comparison to the JED, extension searches on search engines like Google don’t contain VEL information. Extension developers with security vulnerabilities are not held responsible because of this low visibility. By allowing quick and easy access to VEL information from a JED listing page, extension developers will be encouraged to react more quickly, and code more responsibly with security in mind.

Reduced J.org Technical Debt

Maintaining a Joomla site is a huge amount of effort for any team. Updating extensions, updating Joomla, etc… all require a ton of effort. By removing the VEL, the joomla community allows the VEL team to be more productive with managing VEL information, and spend less time on website maintenance.

+100

But

Having an extension’s VEL history log within the record would increase usefulness and functionality to install from web users considerably.

How would that help? All code has bugs and some bugs are security issues. (assuming that any extension with an unresolved vulnerability is not listed)

How would that help? All code has bugs and some bugs are security issues. (assuming that any extension with an unresolved vulnerability is not listed)

It would serve as nothing more as a timeline of events that have taken place. For example:

• Jan 1st extension had sql injection
• Jan 1st developer released update with ~6 hours of notification

This informs potential users of the developer's reaction time and attentiveness (or lack thereof) to issues.

That's pretty unachievable then as in most cases no one other than the developer knows the full timeline.
Example.
10:01am Developer reports a vulnerability
10:02am Developer releases an update
What you dont know (and cant know) is that the vulnerability was reported to the developer 4 months earlier

I'm thinking it would be more related to a 3rd party reporting the vulnerability (via a JED form?), which sends notification to JED/VEL sub-team, and the developer (creating the first portion of the record).

The second portion is achieved when the developer confirms and updates the version number in the ext listing.

PS: use https://giscus.co/ for gist notifications!

Yeah but not seen that happen (doesn't mean it doesn't happen) as its normal for a vulnerability to be reported to the dev first and only made public elsewhere if the dev doesn't react.

Plus it is a little unfair on the honest ext devs who say they fixed an issue compared to others who hide it in a changelog as an improvement. Some of the worse vulnerability I've seen in joomla extensions never appear on any list or changelog.

You could call it 'reported vulnerabilities' as opposed to developers who find, fix, and report vulnerabilities without a 3rd party report. Developers can add a "security fix" to their changelog without having a "reported vulnerability" show up (good for honest devs).

But, with the report record, dishonest devs are held accountable because there's proof someone reported something (obviously published after confirmation), so they have to answer to that with a release to "close" the pending issue.

I do not understand why you are so determined that this is a good idea. What purpose would a 'timeline' of past vulnerabilities serve? If an extension has a lot of fixes what would that signify? That the developer has been conscientous and reported all the necessary fixes and therefore it should be trusted? Or that it is a bad extension, because the developer has had to issue many fixes to his/her appalling code? Or simply that it is a popular extension, so many people have looked at the code and spotted vulnerabilities?

I do not think that it answers a question that many users would actually ask. I suspect what JED users are mostly interested in is whether the current version has known vulnerabilities, not what has happened with past versions. That they ought to be able to tell by whether or not the extension is currently published in the JED. There I do think that you have a point, there sometimes could be better communication between the JED and VEL, so that currently vulnerable extensions are more quickly unpublished, and also that updated extensions are more rapidly re-published. But I do not think that this requires merging of the two teams, just better channels of communication.

The other thing that users probably want to know is whether the extensions they have installed on their site are listed on the VEL. We are actively working on a simple VEL API which can be used to answer that question.

I do not think that merging the VEL with the JED would increase its visibility, I suspect that rather the reverse would happen, given that the VEL is quite a small and specialist team, it could well disappear from view altogether.

As far as the monitoring of non-JED extensions is concerned, the VEL does not exist to support either JED or non-JED developers, but to provide an independent service to users of the extensions. In order to accomplish this we believe that it is right to list any extension which might be in use on a Joomla site, including non-JED extensions, and also sometimes template frameworks, which would never be listed in the JED. We do not provide any kind of advertising for the extensions covered, so I do not see why you think that tracking them is somehow providing a service to the developer - it is not. In fact there are quite a few developers who would probably rather that the VEL ignored them.

I do not understand why you are so determined that this is a good idea.

I wrote a very clear post on why I think it's a good idea...

What purpose would a 'timeline' of past vulnerabilities serve? If an extension has a lot of fixes what would that signify? That the developer has been conscientous and reported all the necessary fixes and therefore it should be trusted? Or that it is a bad extension, because the developer has had to issue many fixes to his/her appalling code? Or simply that it is a popular extension, so many people have looked at the code and spotted vulnerabilities?

It could in theory imply all of those things. That's up to the consumer to decide, I'm just saying it would be a nice, transparent and open way of presenting the facts.

I do not think that it answers a question that many users would actually ask. I suspect what JED users are mostly interested in is whether the current version has known vulnerabilities, not what has happened with past versions.

You could be right. the only thing that matters is whether or not the current version is vulnerability free. BUT, the length of time it takes for a developer to release an update to a security issue DOES demonstrate their attentiveness. It demonstrates that they care about their users and keep their best interests at heart. The only time that such a log would be bad for a developer would be if they take a long period of time to resolve issues. And quite frankly that's a good thing to know about.

That they ought to be able to tell by whether or not the extension is currently published in the JED.

Fair enough, extensions with known issues shouldn't be visible on the JED. But why manage that information in two places? If its vulnerable and we know it, we should track it ON the JED, where the publishing / unpublishing happens. Not on a completely different website. It should all be in one, single unified place.

There I do think that you have a point, there sometimes could be better communication between the JED and VEL, so that currently vulnerable extensions are more quickly unpublished, and also that updated extensions are more rapidly re-published. But I do not think that this requires merging of the two teams, just better channels of communication.

I will never be convinced that two websites are a more effective solution than one in this case. A single source for managing all of this would be far better for productivity, and for Joomla's users. What logical arguments are there for two teams? How is that better from a productivity standpoint? How does two sources of information help Joomla users? Please tell me how that is better.

Better communication could be established by merging the teams, putting them all in the same room, site, and working with the same dataset. Thats the holy grail we should be going after. Tell me why that doesn't make sense.

The other thing that users probably want to know is whether the extensions they have installed on their site are listed on the VEL. We are actively working on a simple VEL API which can be used to answer that question.

Glad to hear an API is coming, but its a completely new set of endpoints that could have just as easily, and more efficiently lived under the ALREADY EXISTING JED API. So we've now done the same work twice, yet again.

I do not think that merging the VEL with the JED would increase its visibility, I suspect that rather the reverse would happen, given that the VEL is quite a small and specialist team, it could well disappear from view altogether.

Unfortunately I think you're wrong - if the JED features things like "reviews" and "download", I think just as fitting a button would be "Vulnerabilities"

You can't possible argue that something like this:
https://www.evernote.com/l/ACrTQyv9JedHxrHS_o2od7wUlUyZcl47atEB/image.png

Is LESS visible than the current solution. It adds the button to every single JED extension (+9000 links!).

As far as the monitoring of non-JED extensions is concerned, the VEL does not exist to support either JED or non-JED developers, but to provide an independent service to users of the extensions.

It should support the JED because that is where most of the users of Joomla go to add extensions to their website. Extensions can live without the VEL, the VEL needs extensions, and the JED is the number one source for those extensions. Those should be coupled extremely tightly, to make joomla better for everyone.

Joomla users don't care about the internal politics of Joomla teams, or people playing nice to work together, they care about a simplified and usable infrastrucutre to publish content.

In order to accomplish this we believe that it is right to list any extension which might be in use on a Joomla site, including non-JED extensions, and also sometimes template frameworks, which would never be listed in the JED.

Unfortunately this conflicts heavily with the decision that was made years ago in the Joomla community. You're essentially saying that decision was either wrong / incorrect, or that regardless of that decision you don't care about the decisions the Joomla community made, and you're just going to do your own thing anyway. That's a huge disrespect to the community who made a decision for GPL and to only support GPL extensions on our property websites.

We do not provide any kind of advertising for the extensions covered, so I do not see why you think that tracking them is somehow providing a service to the developer - it is not. In fact there are quite a few developers who would probably rather that the VEL ignored them.

This has nothing to do with advertising, except that it does look good to be fast to respond to security issues. (Having the vulnerability is just life, how fast you deal with it is how you determine developers who care from those who do not).

The JED is the only thing we have to hold developers accountable. The JED IS the stick. Being unpublished from the JED is the only punishment that one gets from having a vulnerable extension. So tracking non-JED extensions is pointless in that regard as well. You can't do anything to them anyway.

Again, we are not 'supporting' the extensions listed in the VEL it has absolutely nothing to do with any decision made in the Joomla community regarding GPL/non-GPL extensions - we are supporting the users. We are not out to 'hold developers accountable', that is not the purpose of the VEL. Your argument that we are somehow disrespecting the community is frankly insulting and ignorant. Note that the VEL is an official Joomla site, hosted on joomla.org, doing what it was set up to do.

Better communication could be established by merging the teams, putting them all in the same room,

Seriously? You clearly have no idea how Joomla teams actually work. I will give you one good reason why it does not make sense: are you going to pay the international travel costs? Plus who is going to pay for our time to do this? We are volunteers, we have our own work already. I have never met anyone in the VEL, in spite of working with them for the past couple of years.

And the suggestion that this is somehow about internal politics is equally insulting. We work hard at something we think is worth doing, and I do my best to stay out of politics. I find your attitude arrogant and offensive.

I want to try to be a little clear and explain some of the words / intended meanings behind what I said, because I think perhaps there's a language barrier here. I'm not being prentetious when I say that either, I just am trying to communicate clearly:

Again, we are not 'supporting' the extensions listed in the VEL it has absolutely nothing to do with any decision made in the Joomla community regarding GPL/non-GPL extensions - we are supporting the users.

When I said "supporting", I'm not referring to ads, or you "helping" or "hurting" an extension developer. I mean support as in, the community has decided that the Joomla project will only address, and reference, extensions that follow our community-agreed-upon rules.

We are not out to 'hold developers accountable', that is not the purpose of the VEL.

I didn't say that your job was to do this. Its actually the JEDs job to do this, they're the only ones who can do it. But the VEL is a subset of doing that. When the VEL finds a vulnerable extension, that team goes to the JED, and the JED will un-publish the extension. So the VEL is involved in holding extension developers accountable in that way.

It may not be the VELs primary purpose, but it is a task that the VEL team does do. You can't dispute that fact.

Your argument that we are somehow disrespecting the community is frankly insulting and ignorant. Note that the VEL is an official Joomla site, hosted on joomla.org, doing what it was set up to do.

I apologize if you found this insulting. I didn't mean for it to be insulting; I'm just pointing out the inconsistency. Let's look at some other Joomla.org properties:

• JED: does not allow non-GPL extensions to be published.
• Forum: does not allow commercial advertising, or linking to certain extensions
• Magazine: does not allow commercial-like blog posts, and probably wouldn't allow linking to non-GPL extensions.

The Joomla community culture is one that is designed to only deal on-property with those who've agreed to play by our rules. The VEL is inconsistent in this way when it publishes information on non-GPL extensions. Don't you agree that the VEL behaves differently that the other Joomla.org sites in this way?

Seriously? You clearly have no idea how Joomla teams actually work.

Now it's my turn to be a tad offended. I actually do know how Joomla teams work. I have been a member of the following teams:

• Joomla GHoP: 2009
• Joomla Documentation: 2009 - 2012
• Joomla Resources Directory: 2012 - 2015
• Joomla Framework: 2013 - 2015
• Joomla GSoC Co-Admin: 2010, 2011
• Joomla GSoC Lead Admin: 2012, 2013, 2014

I have a lot of joomla team experience. Please don't write my opinion off here - It comes with nearly 5 years of experience of contributing to Joomla in various ways. I know Joomla has politics, and I know there are politics in the VEL. Let's not pretend there aren't.

Plus will give you one good reason why it does not make sense: are you going to pay the international travel costs? Plus who is going to pay for our time to do this? We are volunteers, we have our own work already. I have never met anyone in the VEL, in spite of working with them for the past couple of years.

I think this is where the language barrier may have come in -- When I said "in the same room", I didn't mean literally. I meant figuratively, like a chatroom (Glip), or mailing list, basically any form of team-based communication. I'm sorry if I confused you there!

I totally get that you're volunteers -- I wish you could see that I'm actually campaigning to make your job easier by reducing your responsibilities, increasing productivity, and giving you direct access to the most of the data you actually work with on a regular basis. I'm really trying to help! Please hear me -- I'm just trying to be helpful. Not harmful.

Please believe me when I say, I'm not arrogant, if you felt something I said was arrogant, then I sincerely apologize. I'm trying to fix something I see as a problem, and I want so desperately for you to see that I want the best for everyone: the users, the JED, and the VEL team.