Krzysztof Cieślak 💙💛Feb 2, 2023

I'm excited to finally talk about some pretty cool stuff I've been working on for the last few weeks.

🧵 We've enhanced #HeyGitHub (@code AI voice assistant by @GitHubNext) with some new conversational capabilities.

🧠 With that change, #HeyGitHub can now answer general questions related to software development, generate and edit code, and control the editor while seamlessly following the conversation and understanding its context.

#ArtificialIntelligence
#AI
#DeveloperTools
#GitHubNext

aegilopsJan 7, 2023

@maxd seen "Hey, GitHub!"  yet?

That adds voice control to VSCode, and lets you use CoPilot too. You can sign up to the tech preview:

https://githubnext.com/projects/hey-github/

Sorry to hear about your RSI, hope it eases up  

#VSCode #CoPilot #GitHub #AssistiveTech #VoiceControl #HeyGitHub #a11y #Accessibility

GitHub Next | Hey, GitHub!

GitHub Next Project: Write code without the keyboard. Difficulty typing? Use your voice to code without spelling things out by talking with GitHub Copilot.

GitHub Next
Jim SalmonsDec 9, 2022
Yo Albert @wunderalbert - W/o a public #API, non-GitHub #AI/#ML devs can’t integrate #GitHubCopilot & #HeyGitHub into #CoT, #RLHF, Model-Switching, Prompt-Selection/Sequencing, etc. research. E.g., LangChainAI is on 🔥. See integrations: https://langchain.readthedocs.io/en/latest/integrations.html, & repo: https://github.com/hwchase17/langchain). I urge you to discuss w/ Oege & other Next Hubbers. 🙏
Integration Reference — LangChain 0.0.32 documentation

Jim SalmonsDec 1, 2022
OMGaia Albert @wunderalbert! Have you ✅d’out this paper? 😱👍👏 “MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning“ 🤓🔥 The LangChain folks have taken a 1st cut at this in their Agent subsystem. This is what we need for a role-based `CoT Dialoguer` Agent in #HeyGitHub. 😎
https://arxiv.org/pdf/2205.00445.pdf
Show threadJim SalmonsNov 29, 2022

Hi Albert @wunderalbert,

Why UI/UX framework support for #HeyGitHub voice-coding productivity boosting?

With decades of experience on dev teams of all kinds, I see three user profiles of interest:

1. On large-project enterprise teams, the UI/UK coder tends to be a junior non-engineer designer. These folks can use all the help they can get to contribute to the team.

2. On small consultant engagement-based teams or small entrepreneurial teams, the senior engineer types have to wear many hats. For them, writing UI/UX code is a distraction when they would prefer to work on the actual problem solution code.

3. A third type, which I think fits me, we are thought-leading POC prototypers who are expert at nothing but just need to get our ideas alive for others to see the potential of our concepts. 🤓

In all three scenarios, while the reasons vary, the value of boosting UI/UX developer productivity is clear.

Show threadJim SalmonsNov 28, 2022
Quick update Albert @wunderalbert - My Zoom with Oege is rescheduled to Thursday. So looking forward to this. ITMT, Harrison Chase and the LangChain contributors have been on 🔥 this past week! 😱👍 The addition of the Agents and Memory features seem perfect for exploring the ideas in my #HeyGitHub #Copilot articles. 🤓 LangChain docs: https://langchain.readthedocs.io/en/latest/
Welcome to LangChain — 🦜🔗 LangChain 0.0.117

Jim SalmonsNov 27, 2022
Yo Albert @wunderalbert - Per my ‘Thoughts on Agency’ #HeyGitHub article, the addition of #ConversationChain, #Agent, and #Memory features in #LangChainAI could be gr8 way to explore adding “Sherlockean smarts” into Conversational #Copilot focused on support for building apps based on popular UI/UX frameworks.
https://twitter.com/sjwhitmore/status/1596213155210096640?s=12&t=Khpgtpee1al2eccE5duSSw
Sam Whitmore on Twitter

“🧠 This week I worked with @hwchase17 on adding ConversationChain to @LangChainAI. A conversational agent needs to remember what you've said to it so far and be able to draw on that for further responses. We added an optional Memory module to the Chain class to achieve that.🧠”

Twitter
Show threadAlbert ZieglerNov 27, 2022

Hey Jim!
@Jim_Salmons

Thanks for those kind words, I'm super happy to hear #HeyGitHub is making a difference!

I wasn't aware of your articles yet, they look super interesting! They're going right to the top of my reading list.

Give my best to Oege on Tuesday!

Jim SalmonsNov 26, 2022
Hi Albert @wunderalbert 🤗 TY4Follow. The #HeyGitHub work you are doing w/ Rahul Pandita & the Nexters is so important to us #DisabledDevelopers & #DisabledSTEMstudents. Your expertise is precisely the knowledge I need to understand to pursue ideas I have written about in my last 3 articles at https://bit.ly/Copilot4DisDev. Most recent on Agency #CoT etc. https://bit.ly/agency-and-copilot. I am Zooming with Oege de Moor on Tuesday about this. Wish me luck. 🤞✌️
GitHub Copilot for Disabled Developers – Medium

The journal of a 71 year-old, cancer-surviving, and spinal cord injured Citizen Scientist’s mission to see GitHub Copilot Used as a Life-Changer for Disabled Developers.

Medium
Show threadJim SalmonsNov 26, 2022
@spanichella & @wunderalbert - This workshop sounds beyond awesome!😍 I hope this whole sub-event, or at least Albert’s keynote, is among the hybrid sessions at #ICSE2023. Since #DisabledDevelopers and #DisabledSTEMstudents are a target user group for #HeyGitHub & other voice-coding #AssistiveTechnology, this topic is especially important to us. And unfortunately our various forms of disability can make travel and in-person events challenging.