Quantifying GitHub Copilot’s impact on developer productivity and happiness
https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/
Open AI Codex
https://openai.com/blog/openai-codex/
Twitter:
https://twitter.com/search?q=github%20copilot
https://twitter.com/scoumbourdis/status/1532613467567423492
https://twitter.com/swyx/status/1580390673936658432
https://twitter.com/dannypostmaa/status/1598476315707191296
https://twitter.com/maccaw/status/1586522019927048193
https://twitter.com/willmanduffy/status/1536861352521781248
https://twitter.com/justindross/status/1582902733065289728
Copilot page on GitHub website
https://github.com/features/copilot
GitHub Copilot documentation
https://docs.github.com/en/copilot
GitHub Copilot talk by @mishmanners from NDC Oslo
https://www.youtube.com/watch?v=XX914bV5GmQ&t=20674s&ab_channel=NDCConferences
GitHub Copilot for Business YouTube playlist
https://www.youtube.com/playlist?list=PL0lo9MOBetEGt1PiOOFFveioaDPWI3MFt
GitHub Copilot feedback forum
https://github.com/community/community/discussions/categories/copilot
GitHub Copilot Product Specific Terms
https://github.com/customer-terms/github-copilot-product-specific-terms
-
What is the licensing cost for GitHub Copilot?
Copilot for Individual USD9 per user per month, Copilot for Business USD19 per user per month (needs Enterprise plan, please reach out to your account manager or our sales team) -
What is the minimal software requirement?
It's an extension of your IDE, so it really based on what IDE that you use -
Is GitHub Copilot an extension which can be used in VS Code?
Yes, Copilot is an extension supported in VS Code, VS, GitHUb Codespaces, JetBrains IDEs and NeoVim. -
Does it take care of source code vulnerabilities as well?
You should take the same precautions as you would with any code you write that uses material you did not independently originate, and should take precautions to ensure its suitability. These include rigorous testing, IP scanning, and checking for security vulnerabilities. You should make sure your IDE or editor does not automatically compile or run generated code before you review it. -
Which languages does GitHub Copilot Support?
All languages are supported, however the more common a language, the better represented it will be in our training data and the more robust suggestions will be -
Can it understand the context of complex code projects? How does it understand the architecture of projects?
Copilot for Business uses various elements of the context of your project to provide suggestions, including file content both in the file you are editing, as well as neighboring or related files within a project. -
Have we validated if the code generated by GitHub Copilot is secured. Any tools used to do a security analysis on the generated code base? How is the security assurance maintained for the suggested code by Copilot?
GitHub Copilot has an AI model that filters out some insecure code suggestions, but this is not a substitute for secure software development practices. We recommend taking the same precautions you take with the code written by your engineers (linting, code scanning, IP scanning, etc.). -
Will this take my job? lol
Bringing in more intelligent systems has the potential to bring enormous change to the developer experience. We do not expect GitHub Copilot to replace developers. Rather, we expect GitHub Copilot to partner with developers, augment their capabilities, and enable them to be more productive, reduce manual tasks, and help them focus on interesting work. We also believe that GitHub Copilot has the potential to lower barriers to entry, enabling more people to explore software development, and join the next generation of developers. We are working to test these hypotheses with both internal and external research. -
Can it create unit tests from code?
Yes, Copilot can produce test code in so much as tests look like any other code. By way of example, this demo shows Copilot creating a unit test in Python. Future versions of Copilot are likely to include more robust test creation tools. -
Does it provide code vulnerability report?
Copilot has an AI model that filters out some insecure code suggestions, but this is not a substitute for secure software development practices. We recommend taking the same precautions you take with the code written by your engineers (linting, code scanning, IP scanning, etc.). -
How to scan our code or test our code in co-pilot? How this help security testing team?
Copilot is an AI pair programmer to help you to write code faster. It does not replace static code analysis tools which scan your code for vulnerabilities. However, Copilot will be able to help you to fix the vulnerability when it's detected. If you have requirements for such a tool, I would recommend looking at GitHub Advanced Security. -
Can Copilot write a test case for the function it generates?
Yes, Copilot can produce test code in so much as tests look like any other code. By way of example, this (https://youtu.be/uT4b9HMgKoY?t=54) demo shows Copilot creating a unit test in Python. Future versions of Copilot are likely to include more robust test creation tools. -
ChatGPT does lot of this already. Does this do anything else which chatGPT doesn't do?
Both Copilot and ChatGPT are based on variations of OpenAI’s GPT3, a large language model (LLM) trained on both natural and programming languages. Both generate content based on prompts from the user. The primary difference is in the UX, and type of content returned. Copilot is optimized for code completion, and allows developers to stay focused on the task at-hand by bringing answers directly into the code editor. Developers can request code completions via both natural language (e.g. code comments) and code. Although Copilot can produce natural language suggestions, the model biases for code suggestions.
ChatGPT is a general purpose chat client that produces content based on natural language commands. Because it’s based on the same model and training data used for Copilot, it can produce code suggestions, but doesn’t necessarily bias for code – so developers need to craft their prompt carefully to get helpful results. Also because it’s not integrated with a code editor, ChatGPT requires developers to shift focus. You may find this (https://www.reddit.com/r/AskProgramming/comments/zg7cn7/chatgpt_vs_github_copilot/) Reddit thread helpful for an end-user perspective.
It’s very likely that future versions of Copilot will leverage ChatGPT capabilities in the code editor (e.g. “explain how this method works”)." -
What are the configurations and readiness needs for Copilot?
From a developer perspective, you needs to install the Copilot extension and signup using GitHub credentials. For an Enterprise/Organization Administrator, you needs to enable the Copilot as part of the policy, assign the license to team/member, set the policy on usage of public code, etc. -
I get error "Enterprise managed user accounts are not eligible for GitHub Copilot." How can I try this out?
This indicate that your Enterprise managed GitHub account is not enabled for Copilot. As an individual, you can try Copilot using personal GitHub account and Copilot for Individual. Or request your enterprise admin to enable Copilot for your enterprise GitHub account -
Where is copilot fetching all the code for to process into the AI?
GitHub Copilot is powered by Codex, a version of GPT3 (a generative natural language model from OpenAI) that was optimized by use of source code from publicly available sources, including code in public repositories on GitHub. All languages are supported, however the more common a language, the better represented it will be in our training data and the more robust suggestions will be. -
Does it need an active internet to work?
Yes, you need an active internet connection to recieve Copilot suggestions. -
Is the Codex model pre-trained up to a point in time e,g, Dec 2021, or is it continuously learning from coding community trends?
GitHub Copilot consumes new versions of the Codex model periodically. Although we don't publish a schedule, in practice the model is updated 3-5 times per year. This not only allows us to benefit from model improvements, but also to leverage training data that includes new languages, frameworks, common coding practices, etc. -
Does it provide information on % of code suggested/created by Copilot?
Not today, but thanks for the feedback we will make sure we will pass it to the product team -
Will CoPilot use any technology from OpenAI?
GitHub Copilot is powered by OpenAI Codex, a generative pretrained language model created by OpenAI. -
Is the "faster" observation for developers with certain level of exp. SDE1 or SDE3s?
This is part of the research that we have done. For further details, please see: https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/ -
Can you elaborate about understanding context locally?
Because Copilot is an extension for your IDE, it gets the context on what you have wrote down then will give a suggestion based on that context. -
Any difference between chatgGPT and GitHub Copilot? Both train with same data?
ChatGPT train on the human nature language and Copilot train on the public code that is available in the world (not only in GitHub). -
Will there be an option for Copilot for Education?
Not at the moment. But GitHub Copilot is free to use for verified students, teachers, and maintainers of popular open source projects -
You mentioned for GitHub Copilot to work, the code does not have to be in GitHub. One of the use cases was to accelerate adding unit tests, would this work if existing code is not in GitHub?
Copilot is an extension of your IDE, so the code does not need to be in GitHub. -
Can we operate the GitHub Copilot using voice command also?
Please sign up for the preview of "Hey GitHub!" https://githubnext.com/projects/hey-github/ -
Does it take care of source code vulnerabilities as well?
While our experiments have shown that GitHub Copilot suggests code of the same or better quality than the average developer, we can’t give any assurance that the code is bug free. Like any programmer, GitHub Copilot may sometimes suggest insecure code. We recommend taking the same precautions you take with the code written by your engineers (linting, code scanning, IP scanning, etc.) -
In terms of business purchase for Copilot, how can we share it if we buy it at my workplace?
There is a Copilot for Business for that. Your company needs to be in the Enterprise plan to use Copilot for business. To find out more about Copilot for Business: https://docs.github.com/en/enterprise-cloud@latest/copilot/overview-of-github-copilot/about-github-copilot-for-business -
Any plans to bring it to Xcode? Would make my Apple coding much nicer. Any plans to support Apache Netbeans IDE?
Not at the moment, but keep an eye on our public roadmap: https://github.com/orgs/github/projects/4247/views/1?filterQuery=label%3Acopilot -
How to use copilot within an enterprise firewall? Do we need to enable any connectivity to external apis?
Yes. You can finetune their network traffic using the following endpoints in their firewall rules:
https://github.com/login/device/code
https://github.com/login/oauth/access_token
https://api.github.com/user
https://api.github.com/copilot_internal/v2/token
https://api.github.com/copilot_internal/notification
https://default.exp-tas.com
https://copilot-telemetry.githubusercontent.com/telemetry
https://copilot-proxy.githubusercontent.com -
So based on understanding context, can it do bug fixes on its own?
It can help fixing the bugs. However the developer is the one in the driver seats to accept/reject these suggestions -
We can deploy github copilot on premise?
Not today. To use Copilot users will require to have an account on github.com for Copilot extension to validate the license and an active Internet connection for Copilot to provide suggestions