Houman KhosravaniDec 28, 2023
What do folks think about #substack ? I like the platform’s potential and am exploring content development there - however the CEO has on interviews made comments that are not great in any shape or form, see https://www.theverge.com/2023/4/24/23696530/substack-notes-moderation-bigotry-chris-best - their co-founder then walked those back - does the #mastodon #sigmoidsocial community feel this company has turned the corner and/or the platform is good? Asking honestly.
Substack co-founder says ‘we don’t like or condone bigotry,’ doesn’t explain how Notes will moderate it

Substack CEO Chris Best may have not really answered Nilay Patel’s Decoder questions about whether racist speech would be allowed on its new Twitter-like Substack Notes platform. Last week, co-founder Hamish McKenzie shared a firmer statement in a Substack Note.

The Verge
Fabrice TshimangaSep 19, 2023
I'm probably not good at tooting, but I'm growing bored with #sigmoidsocial, I think I'll move to #neuromatch. I want to discuss both #ml, #ai, and #neuroscience, but the big names of the former are still making noise on X, apart from some researchers that are doing great in the fediverse, but not on this instance. Yet too many seem to need tech-bros (derogatory) to thrive
Techronic9876Mar 7, 2023

Question for #sigmoidsocial

Has anyone seen a comparison anywhere of how LoRA and P-Tuning compare when used with LLMs? I can’t seem to find a comparison of these anywhere

Lora: https://arxiv.org/abs/2106.09685

Ptuning: https://arxiv.org/abs/2110.07602

LoRA: Low-Rank Adaptation of Large Language Models

An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larger models, full fine-tuning, which retrains all model parameters, becomes less feasible. Using GPT-3 175B as an example -- deploying independent instances of fine-tuned models, each with 175B parameters, is prohibitively expensive. We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. Compared to GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times. LoRA performs on-par or better than fine-tuning in model quality on RoBERTa, DeBERTa, GPT-2, and GPT-3, despite having fewer trainable parameters, a higher training throughput, and, unlike adapters, no additional inference latency. We also provide an empirical investigation into rank-deficiency in language model adaptation, which sheds light on the efficacy of LoRA. We release a package that facilitates the integration of LoRA with PyTorch models and provide our implementations and model checkpoints for RoBERTa, DeBERTa, and GPT-2 at https://github.com/microsoft/LoRA.

arXiv.org
Nayef AhmarDec 21, 2022
#SigmoidSocial #AIart #Olphas
Michael DennisDec 20, 2022

If anyone here on #SigmoidSocial is interested in understanding the effects our #AI or #RL systems have on society, I would encourage you to check out the communities Tom is organizing (two of which are linked below)

The design decisions and research directions of today, are fixtures and industries of tomorrow. It's always great to find spaces where these considerations are discussed openly and seriously

[https://sigmoid.social/@tkgilbert/109547206519360129 | CHI workshop]
[https://sigmoid.social/@tkgilbert/109538077146112536 | PERLS reading group]

tkgilbert (@[email protected])

Do you have policy ideas for societal-scale AI? AND want to design for new capabilities? Consider submitting to our #CHI2023 workshop: "Designing Platform Technology and Policy Simultaneously". More info here: http://designpolicy.one/ We are soliciting short position papers (1-2 pages) and short research papers (4-6 pages). Deadline is February 23, 2023--plenty of time to put something together!

Sigmoid Social
Michael DennisDec 18, 2022

Anyone on #SigmoidSocial experimenting with open source recommendation systems for mastodon? With all the #ai folks here, this would be the place to experiment!

https://mastodon.mit.edu/@dhadfieldmenell/109537318540205247

Dylan Hadfield-Menell (@[email protected])

@[email protected] @[email protected] I don’t think there is anything for this. It seems like this is the right time to be building up options. An open source recommendation system could be very interesting

mastodon.mit.edu
Marcel FröhlichNov 27, 2022

Besides Patreon, @thegradient now additionally set up

https://ko-fi.com/sigmoidsocial for donations.

Advantages: Supports both monthly and ONE-TIME payments AND there are NO FEES. All the money you give is for sigmoid.social.

I guess there are no badges for one-time payments, but optionally it should be possible to activate a top supporters list on ko-fi for some appreciation.

Go, give some $. Setting up an account takes 5 minutes.

Please, share!
#sigmoidsocial

Raja Farrukh AliNov 21, 2022
Why doesn't sigmoid.social show up as a server in the Mastodon App? #sigmoidsocial
Show threadeepNov 21, 2022
@andrey @thegradient Hey #SigmoidSocial team, maybe consider adding this in the server announcements, or the site-level blurb? 'In the web client you can click the control panel icon in the upper right over a hashtag column and "pin" it. If you click that icon again over a pinned column, you can choose "local only".' This seems like it could be a good way for people to get engaged in #discussion and #question answering relevant to this instance, if you could advise them as such. #mastodon
Show threadeepNov 21, 2022
@andrey I don't know if you figured this out already, but in the web client you can click the control panel icon in the upper right over a hashtag column and "pin" it. If you click that icon again over a pinned column, you can choose "local only". This seems like it could be a good way to get discussions and questions going on just this server, if people would start doing this. #mastodon #question #discussion #SigmoidSocial