Machine Learning & Artificial Intelligence

ML

Are you curious about the latest buzzword in the tech world?
Have you been wondering what machine learning is?
Let us take you on a journey to understand the basics of this revolutionary technology and its potential to revolutionize the future.

mL (machine learning) is a subset of artificial intelligence and follows the theory of computational training, to understand core principles through computer science and statistics. These training methods can be broken down to supervised, unsupervised and reinforcement learning.

If you’re interested in learning the basics of Machine Learning, then you’ll need to have a sound understanding of Computational Theory and the Basics of Computer Science. This involves learning the fundamentals of programming languages (python), algorithms, and data structures. Knowing these topics will give you a strong foundation to build upon as you dive deeper into Machine Learning; you will also need to understand the mathematics behind Machine Learning-algorithms, such as linear algebra, statistics, calculus, and probability. Additionally, you will need to be well-versed in the different Machine Learning / ML techniques and tools, such as neural networks, decision trees, support vector machines, and deep learning. Finally, you’ll need to understand the various Machine Learning frameworks available, such as Keras, PyTorch, TensorFlow, and Scikit-Learn.

For Python

Information

This documentation is a reference guide to the vast realm of machine learning and artificial general intelligence with examples, concepts and libraries to help you get started! We want to create this whole page as a one stop shop for all your mL needs xD!

AI

Artificial intelligence is an umbrella phrase that encapsulates various fields within computer science, mathematics, philosophy and information with the goal to emulate natural intelligence display by humans and animals. Since Artificial intelligence (AI) is a broad and multidisciplinary field that encompasses various domains, its primary objective is to create systems and machines (physical or virtual) that can perform tasks that would traditionally require human intelligence. These tasks include learning, reasoning, problem-solving, perception, language understanding, and even physical actions.

The roots of AI can be traced back to ancient myths and stories about artificial beings endowed with intelligence. However, the formal foundation of AI as a scientific discipline was laid in the mid-20th century with the advent of digital computers. The term “artificial intelligence” was coined in 1956 during the Dartmouth Conference, which is considered the birthplace of AI research.

During the birthplace, John McCarthy, a mathematics professor, stated that the conference was: “to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”

Scope

The dream for many programmers, scientists, engineers and humans would be to create an entity that could scale past our natural intelligence.
This is a task that would define the 21st century and push the upper limits on humanity, naturism and metaphysics into the next industrial intelligence revolution.

GPT

GPT , currently known as GPT-3, stands for Generative Pre-trained Transformer with the number representing the generation via version control and is a neural network machine learning model
GPT-Neo
Section titled “GPT-Neo”
- Official Github Repo
- We should note that the team, EleutherAI, are no longer maintaining the gpt-neo and their repo is currently in archive mode. However below is the gpt-neox, which is still being actively maintained as for Oct 2022.
- The GPT-Neo may have been an extension of GPT2 but changes to the layering.
GPT-NeoX
Section titled “GPT-NeoX”
- For GPU, we suggest GPT-NeoX, Repo Here
GPT4All
Section titled “GPT4All”
- What I enjoy about this software is that it is really easy to install and use, plus it requires very bare metal resources.
- WebUI for GPT4All written in Flask (Python) by Nomic AI, Repo Here
PyChatGPT
Section titled “PyChatGPT”
- Official Repo. PyChatGPT is an on-going API written in Python to help scale and integrate ChatGPT to various applications / eco-systems via TLS.

GROQ

Groq is an innovative company specializing in AI and machine learning (AI/ML) solutions, known for developing custom hardware designed to accelerate AI/ML workloads. Their hardware architecture, optimized for high performance and efficiency, enables rapid data processing and analysis, making it ideal for complex AI/ML tasks. Groq offers a free API that allows developers to leverage their powerful hardware infrastructure, providing an accessible entry point for integrating advanced AI capabilities into various applications without incurring initial costs.

Groq CookBooks

The Groq Cookbook Examples provide a comprehensive collection of practical guides and code samples to help developers harness the full potential of Groq’s AI/ML hardware. These examples cover a wide range of applications, from basic model deployment to advanced optimization techniques, demonstrating best practices and efficient workflows. For instance, developers can find detailed instructions on using Groq’s hardware (via their API) to examine medical documents, enabling efficient processing and analysis of large volumes of medical data for improved diagnostics and patient care. Additionally, there are examples of utilizing Groq’s capabilities to create sophisticated stock market tools, helping analysts and traders make data-driven decisions with enhanced speed and accuracy. Each recipe is designed to be easily followed, enabling users to quickly integrate Groq’s powerful capabilities into their own projects, accelerate development, and achieve superior performance in their AI/ML tasks.

LLaMa

LLaMa, short for Large Language Model Meta AI, is a cutting-edge foundational large language model developed by Meta, formerly known as Facebook. This state-of-the-art model is designed to assist researchers in advancing their work in natural language processing (NLP). By leveraging LLaMa, researchers can push the boundaries of NLP, exploring new possibilities and enhancing the capabilities of AI-driven language understanding and generation. Meta’s commitment to innovation in AI is embodied in LLaMa, providing a powerful tool for the scientific community to develop more sophisticated and effective NLP applications.

Stable Diffusion

Stable Diffusion is a python-based latent diffusion model that performs image generation through deep learning.

Waifu Diffusion

Waifu Diffusion is a specialized adaptation of stable diffusion models, finely tuned to generate high-quality anime and manga-style imagery. This model focuses on “weeb” content, catering to the aesthetic preferences and cultural nuances of anime enthusiasts. By leveraging advanced stable diffusion techniques, Waifu Diffusion excels in producing visually appealing characters, scenes, and artworks that resonate with fans of Japanese pop culture. Its precise fine-tuning ensures that the generated images maintain the distinctive art style and charm characteristic of popular anime and manga, making it an invaluable tool for creators and fans alike.

The official Github repo for Waifu-Diffusion.

Stable Diffusion WebUI

The official repo for AUTOMATIC1111 Stable Diffusion WEB UI

Fast Stable Diffusion

Fast-stable-diffusion Notebooks offer a streamlined and efficient way to utilize various advanced AI tools for image generation and customization. Integrating Automatic1111 (A1111), ComfyUI, and DreamBooth, these notebooks provide a comprehensive platform for users to experiment with and refine their stable diffusion models. A1111 facilitates the management and deployment of these models, while ComfyUI offers an intuitive interface for seamless interaction. DreamBooth further enhances the capabilities by allowing fine-tuning and personalization of the models. Together, these tools enable users to create high-quality, stable diffusion images with ease and precision.

Prompt Engineering

Prompt engineering theory covers a wide range of different GPT concepts, including examples and short cuts to generate the right style of questions and content.

Engine

The Prompt Engine is a versatile and structured framework designed to define, manage, and execute various prompt-based tasks. It is specifically built to handle a wide range of scenarios in machine learning and artificial intelligence applications, offering a high degree of flexibility and precision. The Prompt Engine is defined using a robust schema in TypeScript, leveraging Zod for validation.

Core Aspects:

Name: Each prompt is uniquely identified by a name field, which allows for easy reference and management.
Description: Provides a detailed explanation of what the prompt is about, helping users understand its purpose and context.
Items: An array of strings that lists the key points or examples relevant to the prompt. This helps in outlining the scope and content of the prompt.
Task: A textual description of the specific task that the prompt is intended to perform. This includes detailed instructions and objectives, ensuring clarity in execution.
Tools: An optional array of tool definitions, where each tool is defined by its type, name, description, and required parameters. This allows the prompt to specify which tools or functions are needed to complete the task.
Output: Specifies the expected output format, which can either be text or json. This ensures that the results of the prompt are returned in a predictable and usable format.
Pathways: A structured object that defines the flow of actions based on user inputs or conditions. Each pathway contains a prompt and a set of conditions that determine the next action, enabling complex decision trees and logical flows.

Roles

Common role examples for text transformers:

Act as a javascript console
Act as an excel sheet
Act as a HR interviewer
Act as an advertiser
Act as a publisher
Act as a music teacher
Act as a relationship coach
Act as a World of Warcraft player and limit the response to 50 characters

Warning : Not all text transformers will let you assign roles, as it might create a security issue / risk.

Chaining

Common Terms include: Chain-of-Thought, Chained Prompt.

Rewrite Prompt Example

Start by giving it this task:

Task: Rewrite user-provided text with the most human like perplexity and burstiness.

Objectives: Prompt the user to input a block of text. Use a language model to rewrite the text with the most human like perplexity and burstiness. Provide the rewritten text to the user.

Rules: Keep burstiness as low as possible Keep perplexity as low as possible Go to extraordinary lengths to make the text more human-like Burstiness is on a scale of 0-100. Perplexity is on a scale of 0-100. Burstiness starts at 0 Perplexity starts at 0 Don’t answer for user

Roles: AI model responsible for rewriting the text. User: Provides the initial block of text to be rewritten.

Strategy:

Approach: Use an AI model to generate the most human like perplexity and burstiness.

Prompt the user to input a block of text.

Provide the rewritten text to the user.

Additional Instructions: Ask the user to input a block of text that they would like to be rewritten. Use a language model to generate a rewritten version of the text that has the most human like perplexity and burstiness. Provide the rewritten text to the user. Be sure to explain to the user the concept of perplexity and burstiness and how it affects the text. Provide an option for the user to adjust the level of perplexity and burstiness if desired. Perplexity and Burstiness start at 0.

Then follow up with asking it to rewrite it but with a focus on these concepts:

Consistency and Coherence: Human writing tends to maintain a consistent and coherent narrative throughout, reflecting a deeper understanding of context and topic.

Creativity and Originality: While AI can be creative, humans often bring unique perspectives, experiences, and creativity to their writing that might be harder for AI to replicate authentically.

Personal Touch: Human writing may include personal anecdotes, emotions, or subjective elements that are reflective of individual experiences and perspectives.

Context Understanding: Humans excel at understanding nuanced context and incorporating it into their writing, whereas AI might sometimes produce contextually inaccurate or mismatched information.

Purposeful Structure: Human writers often have a purposeful structure in their content, carefully organizing information for clarity and impact. AI might generate text that lacks intentional structure.

Inconsistencies: Human writing may include nuanced inconsistencies, errors, or idiosyncrasies that reflect the natural variability present in human language, while AI-generated content may exhibit a higher degree of consistency.

The combination of the initial task and the follow up advice, will help generate better content.