“Why has the Composer repo got password protection? Don’t they know this is open source?” In case it is not clear to all, here is why.
Improving the Merchant Experience
Magento CE is open source – anyone can grab the code from GitHub. The password on the Composer repository is not to stop that. Magento is committed to keeping Magento CE open source. But a goal is to provide a great experience for merchants through the new Magento Marketplace (not fully released yet, due Q1 2016). The authenticated Composer repository is the best option identified so far to achieve a seamless experience for merchants (after the initial authentication step).
First it is important to remember there are different classes of merchants: those that want to spin up a simple store, install a few extensions and a theme, and get on with it; and those who have a more serious development team internally or use an external partner. Both are important to Magento. In this blog post, “merchant” usually refers to the class of merchants that don’t really want to be technically savvy, but which are a significant part of the target market for extension developers. They want a simple and painless experience. “Developers” is used for the more sophisticated class of site developers.
Once the new marketplace is fully live, what will happen is the Composer repository will include everything you have “checked out” into your Composer repo. That is, each merchant account gets its own private repository that will include only extensions that the merchant wants present.
For free extensions (and themes), that means rather than flood a merchant’s Composer repo with everything free, merchants will search through extensions and themes via the online store experience, then “check out” those extensions (and themes) they want to have in their repository. Obviously merchants won’t have to pay for free extensions, but they will have to go through checkout in order to tell the Marketplace to put those extensions into the merchant’s Composer repository.
For paid extensions, the checkout process will of course involve paying for the extension. The Magento Marketplace provides a single place a merchant can purchase extensions made available via the marketplace. It also provides a useful way to remember which extensions you have purchased, even if you are not currently using them. (Purchasing an extension does not automatically add it to your site – it just adds it to your Composer repository. You must then take an additional action to add it to your site an configure it.)
So how to ensure you only see things you checkout and protect purchases from access by other users? We use the Composer authentication capability to do this. In order to make Composer prompt for credentials, we require authentication to access the packages.json file in the repository. Once we know the user’s identity, the repository can return a packages.json file listing only the extensions and themes that user has access to.
Right now, until the Magento Marketplace is fully available, most people will only see CE in the repository – no extensions yet. (EE may be present as well, but I am not sure if the authentication has been set up yet – if not, it will be soon.) This causes the current feeling of “authentication seems like a waste of time”. It won’t be once extensions are available through the Marketplace, and even in the short term it is used to determine if you get access to EE or not.
Oh, and patches will be made available via the Composer repository for CE, EE, and extensions, providing a single place to go to locate and install patches. Again, authentication is used to determine which patches a merchant should get access to.
For the technically curious, behind the scenes there is only one real repository. We just prevent access to free things the merchant does not want and paid things they have not yet paid for. The authentication credentials are used to work out what to grant access to.
Improving the Merchant Experience Quality
The main objective of Magento investing in building a new extension marketplace was to improve the merchant experience. This is not just with the installation and upgrade/patching process, but also with the quality of extensions available. The goal is to help merchants find quality extensions that will solve problems rather just than create new ones. To achieve this several things have been done.
All extensions loaded into the marketplace are put through code and security audits. It is only possible to reliably provide feedback on extension quality if the extension is hosted in the marketplace. Otherwise it is too hard to guarantee that the code submitted to Magento for review is the same as what a merchant downloads. Magento needs to provide the download of extensions in order to ensure that the code is identical to what has been submitted for review.
It also has a side effect of making it easier to spot plagiarized code (where one extension developer may “borrow” code from another extension without honoring the license of the copied extension) and making reviews more trustworthy (the marketplace knows who purchased an extension and so can leave a review). This is all only possible by centralizing all activity, forming a closed system where purchase and feedback are all in the one place.
Authentication
When the Composer repository first launched, it used the merchant’s Magento user id and password. But this was stored in a file on disk and people were understandably not happy with this solution. More sophisticated developers really want it safe to store the authentication information under source code control (e.g. in a GIT repository) to make it easy for team members to share. So the merchant’s Magento username/password has been replaced with a credential system, where a merchant can generate a username/password pair to use, and easily disable it later if it becomes compromised – without having to shut down their whole account. See the developer documentation on how to get these credentials.
Note that the “sync” button in the Magento admin web interface uses these same credentials, and is the normal way a merchant will enter the credentials. They would not use command Composer commands to install and update extensions (like more sophisticated developers).
Magento is also unlikely to officially endorse third party repositories, again because it is too hard to provide guarantees that downloaded extensions are secure and reliable. This is not to be nasty – it is just too hard to provide guarantees on quality and security without controlling the full lifecycle.
As an aside, we had considered having a different URL per merchant (e.g. add merchant user id to the Composer repository URL), but that did not remove the need for authentication as the set of modules used by a site is potentially confidential (even if they are all free). An obviously credentials are mandatory as soon as a paid extension is purchased. The current authentication approach feels like the best all round solution.
Conclusions
Hopefully the above explains why the approach has been taken to password protect the Composer repository. It is necessary to protect paid extensions, but it is also useful even for free extensions as it allows a merchant to have a short list of “approved” extensions for use on projects, rather than having all free extensions listed in tools like the Magento Component Manager.
The end result will be a simpler and more trustworthy environment for merchants.
Note that you can still add extensions from other sources – just add additional Composer repositories to your project composer.json file and add the extensions by hand. If you do this however, be aware you can no longer use the supplied web tools to check for and install patches – currently they only work with the official Magento Composer repository. And if you do so, Magento cannot vouch for the quality or security of such extensions.
Personally, I will be discouraging anyone to set up a separate password free repository. This is not because I personally desire a password protected repository, but rather I think a separate repository has a real risk of undermining the experience of merchants, and potentially the security of their sites. And yes, I know this makes automation a bit of a pain – injecting credentials is harder. But that reminds me of an old security adage – “the opposite of security is convenience”.
Got a better approach? Feel free to comment below! I am expecting a few people to express the opinion “accounts means its not open source”. Just know in advance I understand the sentiment, but until a better solution presents itself passwords feel like the least painful solution once you take the whole picture into account. I am concerned more with improving the security profile of Magento than requiring a password to download the code base in a secure, trusted, and easily patchable way.
Disclaimer: This post expresses how I see the world and does not necessarily represent the opinions of my employer (and by so saying I can get these posts out faster!).
Alan Kent's Blog 
Is there a way (API) to get a list of all packages and versions on the marketplace?
I ask because I use versioneye.com to keep track of new releases.
Not sure – if only because it is not live yet!! I will pass on to relevant internal people.
Two concerns here:
1) I think the ‘step to activate tree module’ will be a ‘composer require’ or something like that. This means that this can conflict with Packagist, don’t?
My suggestion here: create a specific namespace (and reserve it on Packagist), and deliver ALL modules from that inside connect. So we’ll have something like that: ‘ composer require m2-connect/fooman-same-order-number’ and then we all will know that packages from ‘m2-connect’ are official and endorsed.
2) Why we don’t make all free modules enabled by default in the repo? That way people don’t need to specifically ‘buy’ the module from connect. It’s already free, why making someone click on a button, since you’ll be getting usage stats anyway, because the use is authenticated. I guess the only point against here is that people will install modules without reading the module page and seeing important warnings, but that will happen anyway.
(1) I might be missing the point, but packagist poses a security concern as we do not vet its contents, so not sure we want to allow anything to be downloaded from packagist. We are considering mirroring all trusted packages needed from packagist (and running a virus check or something over them). Have not finalized this yet. There is great stuff there, but we don’t want w back door where extension developers can bypass our verification process by depending on a package in packagist instead of something they upload for us to check.
(2) We think giving all users a list of thousands of free extensions in the admin will be a negative interface. Better to browse the large set in the store where there are reviews, quality ratings, etc. and have a short list when in the admin. In talking to people, most said they build up a list of extensions they trust and reuse. Most experts were scared of new extensions being added uncontrolled to this set. We think the new review processes will improve this, but the idea of reusing extensions felt like a good approach. It is also more consistent with the experience around paid extensions where there will be only ones you buy. Getting paid extensions you purchase lost in a list of thousands of free ones did not feel like a good experience. (Composer does not know which were paid or not paid, so cannot filter them etc.)
1) I don’t think it’s a good ditching Packagist. Composer allow us to reuse libraries very easily, and mirroring will just add middleman, delaying access to security updates and new stuff people want. You’ll be listing only trusted extensions from Connect/Admin, so you already checked it’s contents, but if the extension relies on some Packagist library (like Swiftmail, Guzzle, etc), I guess will be better just let it come from Packagist.
Also, you will be checking the extension, you can also check from where the packagist code comes from. ~1.0 in a popular package it’s ok, dev-master from a hideous repo is not. You can filter that before allowing the extension or its updates be available on Connect.
2) Got it, but we still can work with that. Composer uses a “packages.json” file, and the extension list in Connect/Admin certainly comes from a API controlled by Magento, so there is no harm in allowing ALL free extensions in packages.json, but tailoring the API according to the logged user.
I’m thinking as a agency here: to setup a new store, I have my trusted extensions, but I’ll use the merchant’s account, probably with no usage stats so far, no favorite extensions, etc. From composer I’ll just ‘require all the things!’ that I already know and trust, without having to find and checkout again dozens of extensions that I, as a agency, already know what I’m looking for. The merchant, from Connect/Admin, still will be viewing only tailored content. Or, to be more “automatic”, you can just “buy” an extension inside the merchant’s account when we “composer require” that extension. Makes sense?
I mentioned some of the security issues we are exploring in the reply to Ivan (I wont repeat here). Libraries like Guzzle etc are probably not a problem in practice – it is people sneaking other libraries in there with “nasty” stuff in them, since packagist is not vetted. One serious option was to white list “well known” libraries on packagist.
Regarding where the install wizard gets extensions from, currently this is the packages.json file – there is not a separate API today. It is pure composer. If we created a separate API that would be another approach, but would mean a different strategy for free and paid extensions. For now we plan to stick with treated free and paid consistently.
Regarding the system integrator / merchant relationship, we certainly have been thinking about that. We have not settled on a solution yet (lots of other things to do first), but we were considering allowing a merchant to control the granting of access per extension based on generated Composer credentials, so the Merchant can give different access to different system integrators. (Conceptually its like allowing a merchant to create multiple composer repositories.) We also were looking at allowing a system integrator to purchase extensions on behalf of a merchant (or transfer ownership to the merchant after purchase). It was not clear how far we needed to go, so we were going to start simple, get the Marketplace up, and learn and iterate. But we have certainly been talking with system integrators, merchants, and extension developers – and similar patterns emerge from multiple conversations, which is good.
Thank for the feedback!
1) Limiting access to packagist is killing all the idea of using composer. Also, it is not required to limit packagist access to test for security issues, you always can run security checks on extensions with a dependency on packagist package. It is as simple as running a composer install on CI tool, isn’t it? Then after you simply need to check resolved dependency version on install with checksums with checked version on Magento connect. Also, you can replace wildcard dependencies in extensions with automatically checked secure library versions.
In that case in package.json you can control also out-dated insecure versions of well known libraries.
One option we were considering is white listing. There are two use cases – the first is someone wants to use a library on Packagist. Should we do a security vetting of 3rd party libraries to make them usable on Marketplace? Maybe. We would need to spot each new patch release to retest (before anyone else noticed ideally). It is not clear yet if the “ideal security posture” is just overkill in practice.
The second is someone purposely trying to bypass security checks by creating a new library on packagist and referencing it from their extension. In this case they are just trying to bypass the rules – this situation we do want to stop.
We were going to observe submissions for a while to see how extensions use packagist in practice.
Hi Alan,
Thank you for this blog post, and your other posts. And thank you for asking for opinions and encouraging a discussion.
Here’s my opinion from a hosting provider perspective:
1) The idea of magento merchant having their own magento.com user account doesn’t make sense to me as a hosting provider who provides new magento installations. We have many customers who come to us wanting to use Magento for which we install a new copy for them. Many never setup an account on magento.com. Telling them they need to create their own account on magento.com and send us back their credentials is awkward to say the least…
2) Most merchants (I would say 60-70% of our customers in fact) have no idea how to do anything other than manage their admin. So who now owns this magento.com account? Do we as a hosting provider / agency / developer have to make the merchant setup a magento.com account and pass this around between us? Or do we have the ability to setup multiple installations and share them between multiple magento.com accounts? What if then a customer switches agencies/etc and the agency is managing their extensions, can they transfer this to another account? We see thousands of relationships and most are not a 1 to 1, single merchant magento.com account to single installation where the merchant is involved in development. Again, most merchants have no idea other than to manage their admin. Many jump between agencies. Many have multiple developers. Heck we even install extensions for some customers! It sounds to me like we’re all going to have to pass around this customer’s magento.com login and doing so is not very secure at all…
3) I can understand why people won’t like this on an ethical level too. For example, let’s say a merchant purchases an extension. How will you control licensing? Will you “phone home” for license validity checks? If not, what prevents someone from installing a copy of an extension on multiple stores? What if time passes and the merchant is not satisfied for whatever reason, the merchant files a chargeback, and a refund is issued? Are you going to build in access control to revoke this extension and disable it later? So many ethical questions that potentially break open source mentality and there will always be a fear of a failed auth process that downs a production store…
4) Why does the magento.com my account appear like it’s using key-based authentication? Inside My Account on magento.com it says “secure keys”? Inside it says “private key” and “public key”. The composer auth.json file you have to put the public key in for the username and the private key in for the password. That’s very strange…
Lastly, I do like the idea of vetting code quality in extensions and think you have (mostly) good intentions. Although I have my doubts the resources are in place to efficiently and accurately succeed at such a monumental undertaking of high quality reviewing. I hope it works because it would be great for us to have a golden set of extensions for which we can also provide support for in addition to the core. The stability this could bring would make our lives much easier. I REALLY hope you take a closer look at how most magento stores are being developed/deployed and how this extension set is managed. Also, please tread very carefully in the authentication/revocation control processes.
Once again (because it cannot be said enough) thank you for taking the time to write these posts, reading my comments above, and encouraging an open discussion in general!
Thanks for the feedback. It is useful getting different perspectives. We have to pick a route at the end of the day, but its better to do so informed!
(1) For a hosting provider where you set up an initial account, there is not technical reason why the user needs to have an account. While I might not recommend it, you can always download CE, have a local composer repository (or maybe use Toran proxy), and set up the services that way. That is, you as a hosting provider take responsibility for patching and upgrading that customer. (I don’t recommend this, because it sounds like extra work for you.) But for us, as soon as we need the potential of different identities to allow your customers to purchase an extension (and have us remember the purchases for them), that requires identity to be set up. We need to give that user, and only that user, access to the repository. Better, it feels like us providing an API to make it trivial for you to create a magento.com account on behalf of the user during signup feels easier. Note that the initial setup from a ZIP you have downloaded does not require the magento.com account to be created – you could defer this until they want to go to the store and make a purchase. That is, you can provide the registration details later (maybe?).
(2) Do you think merchants want to get extensions and themes or stick with a vanilla site? As soon as they want extensions, are you going to provide a set of themes and extensions? Otherwise I think they will need to access the Marketplace. I think the responsibility is on us to make it as seamless an experience as possible. E.g. create links from Admin to the Marketplace (or have an in-admin marketplace experience) so the merchant can do it all from their web browser just by following links. That certainly is our intent.
We are thinking about the agency/merchant relationship as well. That won’t be there day one most likely, but we understand there are a range of use cases here that we need to ensure we support appropriately. One thing we do need to do here is make sure there are not shared accounts where a group of merchants buy an extension once and share it. That is not fair to the extension developers. Currently (as per separate response to Ivan), we are thinking that merchants need to own extensions, but be able to grant rights to agencies. Agencies probably never purchase extensions for themselves – they just do it as a service for merchants. But merchants need to be able to change agencies if they wish. How to keep this simple is the challenge.
(3) There is a level of protection we cannot afford. There is a refund policy for example – how to stop someone keeping and using the code if they do a refund? At some level you simply cannot stop bad people doing bad things. But you can at least make it easier for good people to do the right thing. No, there are no “phone home” or other schemes planned at present. Frankly, if someone was determined enough to break the rules, they could turn off any such checks. But if you do not purchase, you wont get patches for example. We also can tell extension developers who purchased (and returned) extensions.
You made me realize however I don’t know if it is acceptable for a merchant to use an extension on multiple stores – something for me to check up on. It may be instead of a “repo per merchant/agency” it is really “a repo per site” where you control which agencies can access a site repo. (Maybe!)
(4) The terminology is a little strange to me too, but basically we generate a username/password pair to use so merchants don’t have to save their magento.com account name and password in a file on disk (the composer auth.json file). It makes it easier for a merchante to grant and revoke permissions for agencies for example.
Regarding quality of reviewing, we will be lifting the game – it certainly won’t be perfect. We will however continually try to improve it as we see more extensions and learn. Security audits for example are not going to find *every* security issue – that is just not realistic (unfortunately). So it should be *better*, not *perfect*. Better still has value.
Thanks for the feedback. Always valuable.
Your comment is awaiting moderation.
Alan, thanks for taking the time to run through the rationale and decision making process behind the new design and implementation plans for Marketplace / Connect.
I (and others) may not agree with all of them, and may identify scenarios where this process might run into snags in my specific use cases, but at least the Community can see that these aren’t arbitrary decisions.
I think it’s great that you’re stating clearly the prioritization of use cases (starting with Merchant, then dealing with others) – it helps me understand why we’re seeing features as
currently implemented but also helps me in my planning discussions with Merchants.
I’d request two things, if possible:
1) Please do what you can to make Security Feature #1. I know it’s not easy from a business or technical perspective, and agree that doing so actively impedes feature development, increased costs, and frustrates business users and developer alike. But with online fraud expected to more than double by 2018 in the US as EMV adoption grows, I think it’s one of those “we actually know what’s best for the community as a whole” situations, even if it causes some short term pain / frustration. IMHO, the technical community can work with your teams to figure out solutions & work arounds to ensure that automation / DevOps investments are protected while less sophisticated deployments are made as Secure and convenient as possible.
2) Where possible, continue the background reasoning transparency you outline above but breaking out feature timelines. Although dates would obviously be most helpful, even just a feature prioritization timeline would help planning efforts tremendously. Understanding when each major use case set (i.e. We’re going to complete the features required for scenarios A, B, C for “traditional” merchants, then begin features targeting agencies) is prioritized will help integrators gauge overall status, especially during the early discussions with clients.
I have been excited at the level of openness and transparency from Magento since the initial Github repos were published and am happy to see that trend gaining momentum. I’m incredibly thankful for the effort put forth by the Magento team, the Executive Support so clearly being provided, and the feedback from the #RealMagento community to which you are obviously listening.
Thanks for the feedback. Timelines are often hard – but certainly priorities are to get the Marketplace up, keep it somewhat locked down, learn, iterate. We would rather fix real problems for people rather than imagined. So we think about likely problems, but don’t implement them all immediately. We would rather get real world feedback to confirm it. This does however make it harder to publish timelines and plans – we have ideas, but not commitments until we work out if really a problem (and how big a problem). Onwards!
Again about packagist in reply to Alan about
1. You cannot bypass security checks by Magento if package.json is based on resolved library versions during automatic security checks via composer install, as you receive all dependencies in vendor directory and can validate all code at once, there is no special magic involved. Just skip execution of post-install/post-update commands with needed arguments.
2. This tool might be an interest for you guys on monitoring valnurable dependencies: https://security.sensiolabs.org/
3. It will be a much more bigger nightmare for Magento to manage a list of whitelisted packagist libraries (might be you’ll have some kind whitelisted namespaces for automatic validation pass).
Current way of doing things in PHP community is using libraries as decoupled packages that do their own job good. For instance, if I want to install CSV reader, I would use phpleague csv package, if I want a Yaml parser, I’ll use symfony package and so on. I will not write this stuff myself in extension, as it is an overkill for me as a module developer. You can imagine that there are 1000+ packages for specific APIs, file formats, parsers, writers, etc… How can you whitelist them all?
4. Just a very good example is Microsoft/Azure PHP library, that requires merchant to specify custom composer repository for installing some of its PEAR dependencies. Would you prohibit Microsoft Azure merchants from using Magento automatic upgrades? Would you include Azure libraries into repo.magento.com? I think there might be some legal issues with that.
https://azure.microsoft.com/en-us/documentation/articles/php-download-sdk/
5. How about own PHP libraries of module developer? If I create a library that might be used not only in Magento, but by wider audience, I want to put the library on packagist, I don’t want to ship it with every my module as git module, it is an overkill.
In general, I do like an idea of authentification for merchants, as it gives a good protection of IP, and I fully support full review of extension by Magento. But I wouldn’t disallow extension, just because it is using library from packagist that is not whitelisted, as you can validate composer.lock after install and check all dependencies yourself with static code analysis.
(1) I don’t fully understand sorry. If we allow access to all packages on packagist, then the packages.json file is from packagist which is out of our control.
(2) Thanks – we will have a look. (I don’t recall what tool we use today.)
(3) We have not finalized, but the whitelist if done would likely be based on demand – if someone tries to use a library in an extension, we would add it to the whitelist after review. We would not pre-approve everything on packagist (too big a job). None of the options we are talking about is asking extension developers to merge into their extension. The question is just whether we mirror scanned packages in our repo, or whitelist access to the official packagist.
(4) Thanks for the example – these sorts of issues are great drivers to help us pick between options. Legal restrictions simply frequently don’t care about “developer experience” so limit our choices.
(5) To be clear, we have never said we don’t want to allow multiple packages – it is only how to do it securely and not make it a backdoor for some developers to avoid our validation steps. So I think we are in alignment here. The question is only about how to achieve all the goals at the same time. We have not finalized on that yet.
1) Seems like it was a common misunderstanding, I didn’t mean packages.json from packagist neither. I mean that you can create specific dist package specifications for “require*” sections in extension composer.json based on composer.lock after install execution of your CI tool. You still install package from packagist source, as you are saved by composer shasum check for dist package.
“dist”: {
“type”: “zip”, // tar, etc
“url”: “[url]”,
“reference”: “[gitsha]”,
“shasum”: “[shasum]”
},
As composer prohibits installation of failed checksum check downloads, you are 100% secure that infridget packages is not in the system. And you are safe with checked package
I know already 3 ways to bypass security checks with very evil possibilities when packagist ist allowed.
Can you elaborate on those more? Is it really possible to fool composer if you have –no-plugins –no-scripts option specified during composer install?
I can, but that needs some time to write it all down.
And something makes me guessing, people will still not belive me, so I also need to write a Proof of Concept, which again needs more time.
And then we have a 0 day which ruins everyone’s Christmas, so I would name it The Grinch.
And after an hour of investigating, for the target group Developers, there is a 4th security hole created by the pure existence of packagist, even if you disable it for magento projects.
Why you simply don’t submit a patch to composer as a pull request? If there is a 0day exploit, it must be fixed in it. I think you can get a good appreciation from all the PHP community.
Because its not a simple patch. The problems exists partially by Design and are not directly unknown. And till now, noone really tried to do bad things. Its just the usual security vs usability problem. They chose usability higher, Magento chose security higher.
Regarding the 4th one, a Disclosure with a bit of example code is send to the maintainers. They should be able to find a solution for this one in time.
Alan,
One more question. that just popped up in my mind:
If a solution provider has a set of re-usable modules, but they don’t want to sell those modules as extensions on the marketplace. Those modules should be re-used across different projects, but those modules are not sellable by nature on market place, as they require project specific adaptations and those are solution provider competitive advantage. How to deal with such a situation if custom composer repository is not allowed by Magento. Will Magento allow developer accounts without selling options, but with a control over which merchant is granted access to which module?
Hi Ivan. You can use the command line Composer commands with multiple repositories – it is only the web interface built into the install wizard (Component Manager) that cannot cope with multiple repositories. If you are sophisticated enough to have a system integrator with local custom packages, I think they can handle the command line Composer commands to fetch patches etc. Does that seem reasonable?
Hi Alan!
I perfectly understand the interest of using a composer repository with credentials.
For me, there are two sides, the Developer (and merchant) and the Extensions Provider.
If you are a developer. You don’t want to loose freedom on your actions. You want to use composer like you always do, and you don’t want to provide any credentials!
Of course as a developer you can admit to provide credentials in order to get paid extensions. Seems fair.
If you add too much restrictions for free extensions all of them will move to github and packagist, and you’ll loose control (statistics…).
Why checking on requirements for every Magento extension? If somebody wants to do bad things, they will do it, no matter how, no matter when, no matter what.
Free extensions market has to be free. You can put a stamp on extensions you reviewed and that’s it! If you can’t or if you find something bad in the extension you can remove it or disallow it on Connect, it’s your Connect finally :).
What about paid extensions…
If you are a developer/merchant, my opinion is the same as above.
But if you are an extensions provider you want to be sure that only people who paid can install your extensions. And that’s fair.
In that case yes : you have to use a composer repository with credentials. And please do your review like free extensions, maybe more review, maybe with different rules, but like I said : if a developer wants to implement bad things, he will find a solution.
And don’t forget your community. If the community finds a fuuuuuuu module on the Connect, they will scream! And because we know that you are listening, I’m sure that the bad extension will be removed. Is that true?
I’m not sure to be clear…
But for conclusion:
– free extensions: check the extension, maybe check requirements with whatever you want. Put a stamp if the extension is good (with multiple stamps maybe for multiple levels? even bad levels ;)). And use a full composer repository with all the free extensions. But don’t put any credentials for them, please.
– paid extensions: check the extension, check requirements… check if the extension is following your rules and that’s it. Use a private (like you explained in your article) composer repository only for paid extensions and move on!
And what about merchants who doesn’t have a Magento account… You can still provide the extension in a tarball and explain how to use a composer repository “path”. We do that for years now :).
For the merchants who wants to change their agency… It’s impossible to prevent the bad things about that. You can’t be God.
I mean… actually if a merchant wants to change… they can if the agency if ok with that. Sometimes the agencies pay the extensions and re-invoice them to the merchant. Sometimes they don’t. You can’t control that, and you won’t be able to… ever.
The good… the bad… and the ugly. It’s just… life.
I hope to be helpful. And I’m sure there is more good people than bad or ugly one (I hope).
Thanks for the long and thoughtful comment.
The password protection is so we know the list of purchases to return in the packages.json file. Without it we don’t know what extensions they have purchased (or not).
The credentials also allow you to avoid flooding the UI with thousands of extensions you don’t care about. You only see the ones you are interested in.
The credentials are cached of course, so you only have to set them up once and never type them again.
Given there are developer benefits in the UI (at least there will be when marketplace is fully online), at the moment we are going to stick with the current situation until the store is up and people experience it all. I.e., no need for command line composer commands via store. Soon!
We are an agency, so will have multiple sites installed on our systems that I suppose should be connected to multiple Merchant accounts (otherwise they will all have access to each other’s purchases!). How do we maintain composer authentication for different sites when auth.js appears to be global for a system user?
We are working through this use case internally – current model not enough. Until then, I recommend using merchant account keys.
Thanks. One possible solution seems to be to use an auth.json file at the top level of the repo. This will be used in conjunction to composer.json without having to put the details into a global ~/.composer/auth.json file. Seems to work so far, and as it is seperate to composer.json we can avoid checking it into our repo so it’s secure enough too.
Yes, that was the idea. Put auth.json in project directory.
Is there any way where I dont have to provide Magento 2 EE account access to everyone in team and they could still do composer install successfully ? We like to use composer for our custom modules here is currently it’s setup
1. Git repo containing Magento 2 EE default code with gitignored vendor folder (We call it Core Repo)
2. Git repo containing all custom modules (Custom module repo)
Now in core repo composer.json we add custom module so when developer would do composer update or composer install it will fetch all custom modules and put autoload correctly.
Now problem is if developer is using generic Magento commerce account , composer install wont work for enterprise modules and throws authentication error. We dont want to share Merchant Magento 2 EE account access with every developer in team (By share I do not mean sharing password obviously even if its delegated account access we dont want to do that). As magento 2 ee tarball already contains all required code can there be a way to tell composer not to call for authentication and only fetch custom module repo? this way we will be able to use composer while not have to share account with every dev in team.
Thanks,
Aravi
Local/internal/protected Composer cache like Satis? I can’t see why I wouldn’t work.
I am trying to understand fully, so sorry if I am missing the point. It sounds like you have the EE code in a GIT repo, so your Devs can see the EE code. In which case, the composer repository keys will not give them access to anything extra right? So why not just create a new set of keys for those developers to use. If you give them separate keys, you can delete (revoke) the keys later if you need to.
The keys give them no rights other than to do downloads. If they have access to the EE code in your GIT repo already, then I don’t see the problem of granting them keys.
There may be other hacks you can do, but I don’t recommend it. And sorry if I have not followed your problem exactly.
Oh, if you cloned our EE GIT repo into your own local copy, we never recommend building sites from that. We always recommend downloading via Composer. Otherwise it may bypass compatibility checks of extensions with the code base that COmposer implements.
Thanks Alan for your quick response. there is some internal complications for providing keys to each developers. We will work it out.
I would actually be very interested in understanding. We have plans around future work in key management, but want to make sure it’s really solving problems.
E.g. we are considering having “projects” where licenses for EE and extensions are associated with projects. Then you can have multiple people (MAGEID’s) associated with projects. The keys are also per MAGEID+project. This can help with flows like an SI creates a project, buys extensions for it, adds Merchant as owner later, then Merchant can remove SI from project after work complete. It also gives finer grain control over who can access what in an organization. Would this help or make things worse?
Hey Alan! Thanks for sharing this article. I really like your writing style. Magento is a Wonderful platform for developing eCommerce store. It is cost effective, flexible and Open-source platform. If you look at ratio, almost 70% eCommerce websites are made with the help of the Magento. Magento has some exceptional features that help developers to develop eCommerce websites that are high in functioning, feature-rich and comprehensive. Magento is free, and it is something that offers developers ease and comfort that they are looking for.