Metaroids

    Subscribe to the Metaverse Newsletter

    Join me as we build a network of metaverse and web3 pioneers.

    What's Hot

    OpenAI Hired Developers to Train AI to Replace Developers

    January 28, 2023

    Ex-Google SVP Launched AI-Powered Search Engine

    January 28, 2023

    BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

    January 28, 2023
    Twitter
    Metaroids
    • Home
    • News

      OpenAI Hired Developers to Train AI to Replace Developers

      January 28, 2023

      Ex-Google SVP Launched AI-Powered Search Engine

      January 28, 2023

      BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

      January 28, 2023

      Lawyer Could Face Jail Time for Bringing AI in Court

      January 26, 2023

      Anand Mahindra Sparks Alarm Over Deepfake Videos

      January 25, 2023
    • Learn

      Midjourney Mastery: A Guide to Using Image Prompts

      January 22, 2023

      Design UI Like a Pro: 5 Advanced Midjourney Tricks

      January 13, 2023

      Reddit Moons: An Ultimate Guide

      January 12, 2023

      The ChatGPT Era: How Creatives Can Ensure Longevity & Thrive

      January 7, 2023

      Google Translate vs ChatGPT vs DeepL: Translator Ultimate Showdown

      January 6, 2023
    • Others
      1. Feature
      2. Press Release
      3. Opinion
      4. Lists
      5. View All

      ChatGPT: How AI Can Potentially Impact the World’s Religions

      January 27, 2023

      Humans’ Quest to Decode Animal Languages Through AI

      January 25, 2023

      ChatGPT + Wolfram Alpha: A Super Powerful Assistant

      January 22, 2023

      AI-Powered Sexbots and the Risks They Could Trigger in the Future

      January 21, 2023

      Hyped NFT Trippin’ Ape Tribe to Launch on Solana in Mid-Late May

      May 6, 2022

      Microsoft’s VALL-E May Trigger an Avalanche of Cyber Crimes

      January 18, 2023

      Why Web3 Gaming is Failing & How to Turn the Tide

      December 23, 2022

      Can the Metaverse Work Without Blockchain?

      August 17, 2022

      NFTs are 100% Sustainable: Here’s Why Critics are Wrong

      August 8, 2022

      Best Solana Wallets for NFTs, DeFi, & Security

      October 27, 2022

      Top Move-to-Earn Crypto Apps That Pay You to Exercise

      August 31, 2022

      Top 13 Blockchains for NFTs (Choose Wisely)

      August 16, 2022

      Top Selling NFT Collections of August 2022

      August 9, 2022

      OpenAI Hired Developers to Train AI to Replace Developers

      January 28, 2023

      Ex-Google SVP Launched AI-Powered Search Engine

      January 28, 2023

      BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

      January 28, 2023

      ChatGPT: How AI Can Potentially Impact the World’s Religions

      January 27, 2023
    • Contact
    Twitter Discord LinkedIn
    Metaroids
    Home » An Open-Source Version of ChatGPT is Coming
    An Open-Source Version of ChatGPT is Coming
    News

    An Open-Source Version of ChatGPT is Coming

    By Evan EzquerDecember 31, 2022Updated:January 10, 2023No Comments4 Mins Read
    Share
    Facebook Twitter LinkedIn Email Reddit Telegram

    Philip Wang, the mastermind behind the reverse-engineering of several closed-source AI systems like DALL-E 2, AlphaFold, and Imagen, has just released an implementation of PaLM + RLHF, a text-based AI model that works just like ChatGPT.

    This system combines Google’s PaLM, an LLM (large language model) with 540B parameters (3x more parameters than ChatGPT), and a method commonly known as Reinforcement Learning with Human Feedback (RLHF) to allow for the creation of a chatbot that can do just about anything ChatGPT can, including answering general questions, writing emails, and creating computer code.

    Note that this version isn’t exactly the same as the PaLM model developed by Google a few years ago, but it has a very similar architecture and approach. Wang is well-known for being able to “port” several other famous architectures, hence, he might know a thing or two about emulating these models.

    The Power of PaLm + RLHF

    Since its release, ChatGPT, a fine-tuned version of GPT-3.5, has taken the tech world by storm due to its ability to generate human-like text with high clarity, allowing it to respond in a conversational manner. While it may be a significant advancement from earlier chatbots, many proponents in the AI field have raised concerns over the closed nature of ChatGPT.

    As of today, the ChatGPT model remains proprietary, which means that the public cannot view its underlying code. Only OpenAI truly knows how it works and what data it processes. This lack of transparency can have far-reaching implications and may affect trust from users in the long term.

    Many developers have been eager to build an open-source alternative and now it has finally arrived. PaLM + RLHF is built exclusively for the Python language, and can be implemented for PyTorch. Developers can easily train PaLM like an autoregressive transformer and then train the reward model using human feedback.

    What’s The Catch?

    Well, you can’t use it today yet. So what would it take for it to be available to the public?

    To launch PaLM + RLHF, you will need to compile gigabytes of text taken from various sources such as blogs, social media posts, news articles, e-books, etc. These data are fed to the fine-tuned PaLm model, which will generate several responses. For example, queries like “what are the basics of Economics,” might yield responses like “Economics is the social science that studies…”

    Human volunteers will be employed to rank those responses from best to worst, using the rankings to create a reward model that takes the original model’s responses and sorts them in order of preference, filtering for the top answers to a given prompt.

    However, the process of aligning this model with what users want to accomplish with ChatGPT is both costly and time-consuming, as PaLM has a massive 540B parameters. Note that the cost of building a text-based LLM model with only 1.5 billion parameters can reach up to $1.6 million.

    When Can I Use an Open-Source ChatGPT?

    At the moment, it is unclear how many organizations have the technicals and financial infrastructure to run the implementation built by Wang without degrading their user experience. So far, we have three known players working on this open-source ChatGPT alternative:

    • CarperAI (in partnership with Hugging Face, Scale AI, and EleutherAI)
    • LAION – the non-profit that supplied the dataset used to train Stable Diffusion
    • Yannic Kilcher

    None of them have given any hard dates for release as of yet. But to give you a rough timeline, it took three months to train Bloom, an open-source model that boasts 176 billion parameters. With that in mind, it might take more than 6-8 months for us to see a worthwhile release.

    As Wang AKA Lucidrains says, “good code is just a prerequisite to begin the journey. It will take data, compute, [and ]adventurers to actually set sail…”

    For now, all we can do is wait. In the meantime, it might be a great idea to continue using ChatGPT whilst it’s free. It’s also worth noting that by the time an open clone exists, OpenAI might be far ahead in development. There are many rumors of GPT-4 being ultra-powerful and is set for release in 2023.


    Join our newsletter as we build a community of AI and web3 pioneers.

    The next 3-5 years is when new industry titans will emerge, and we want you to be one of them.

    Benefits include:

    • Receive updates on the most significant trends
    • Receive crucial insights that will help you stay ahead in web3
    • Access whitelist spots of the most hyped NFT drops
    • The chance to be part of our OG community, which will have exclusive membership perks

    Subscribe to the Metaverse Newsletter

    * indicates required


    By signing up, you agree to our Privacy Policy agreement.
    ChatGPT open source PaLM RLHF
    Share. Facebook Twitter LinkedIn Email Reddit Telegram
    Evan Ezquer
    • Website
    • Twitter
    • LinkedIn

    Evan, the founder of Metaroids, is an OG crypto enthusiast and content creator who has explored the world of blockchain, AI, and other cutting-edge tech for nearly a decade. His ultimate goal is to build the best community on the Internet. Up until January 2023, Evan has been writing on Metaroids under the pseudonym Falkris.

    Related Posts

    OpenAI Hired Developers to Train AI to Replace Developers

    January 28, 2023

    Ex-Google SVP Launched AI-Powered Search Engine

    January 28, 2023

    BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

    January 28, 2023

    Lawyer Could Face Jail Time for Bringing AI in Court

    January 26, 2023
    Add A Comment

    Comments are closed.

    Latest Articles
    News

    Ex-Google SVP Launched AI-Powered Search Engine

    By Stark
    News

    BuzzFeed Plans to ‘Hire’ ChatGPT as Newest Content Creator

    By Damocles
    Feature

    ChatGPT: How AI Can Potentially Impact the World’s Religions

    By Damocles
    Metaroids
    Twitter
    • Home
    • About
    • Contact
    • Our Authors
    • Privacy Policy
    • Sitemap
    Privacy Policy and Terms of Services
    Copyright © 2023 - All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.