BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps

•

1 gostou•252 visualizações

Containerization of scientific applications has done a lot to make software more portable and reproducible. Unfortunately, most HPC clusters have been left out of this movement because of kernel requirements and security policy. In collaboration with Björn Grüning representing BioContainers and Vanessa Sochat from the Singularity project, we are bringing thousands of science apps to any cluster that can run Singularity.

Tecnologia

Containeriza*on of scien*ﬁc applica*ons has done a lot to make
so6ware more portable and reproducible. Unfortunately, most
high performance compu*ng (HPC) clusters have been le6 out
of this movement because of kernel requirements and security
policy. In collabora*on with Björn Grüning represen*ng
BioContainers and Vanessa Sochat from the Singularity project,
we are bringing thousands of science apps to any cluster that
can run Singularity.
BioContainers for supercomputers:
2,000+ accessible, discoverable Singularity apps
John M. Fonner, Rion Dooley,
Jacquelyn Turcinovic, MaWhew W. Vaughn
Introduc)on
Singularity Hub
singularity-hub.org
BioConda
bioconda.github.io
BioContainers
biocontainers.pro
Key Capabili)es
•  All TACC HPC systems now support
Singularity images and BioContainers
•  Cyverse supports BioContainers via
Docker on their cloud resources and
now via Singularity on HPC resources
•  Convert your own Docker containers
to Singularity for use at TACC using
github.com/TACC/docker2singularity
Future Work
•  Con*nuous integra*on from BioConda
to TACC / Singularity Hub
•  Enroll as a Singularity Registry on
Singularity Hub
•  Explore GPUs/co-processors support
and custom writable directories
•  On-demand conversion of Docker
containers to Singularity
BioConda has over 2,400 recipes for bioinforma*cs apps thanks
to the contribu*ons of many researchers and developers. The
Cyverse project has adopted Conda recipes as the primary
method for integra*ng new apps into their infrastructure.
BioContainers has automated the process of conver*ng Conda
recipes into so6ware containers and has woven their framework
directly into BioConda. All Docker containers are con*nuously
registered with a Quay.io repository.
Singularity-Hub supports integra*on with GitHub to generate
Singularity images based on git commits. They are extending
their code-base to support ins*tu*onal repositories of images.
TACC has installed Singularity run*me support on its HPC
systems, has converted all BioContainer apps to Singularity
images, and has made them available to XSEDE, TACC, and
Cyverse users. Public, unauthen*cated download will be
available as soon as possible through Singularity Hub.
Summary
Get involved
•  Contribute your bioinforma*cs apps to
BioConda and join the community
•  Try out Singularity at TACC through
either xsede.org or tacc.utexas.edu
•  Chime in on our development eﬀort:
github.com/TACC/docker2singularity
singularityhub.github.io/singularity-
registry/

Mais conteúdo relacionado

Semelhante a BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps

Presentation at the OpenStack Summit in Barcelona, Spain on October 25, 2016. http://bit.ly/os-kub-oci-cncf Containers along with next generation topics such as orchestration and serverless computing continue to draw interest across the application developer and data center operator communities because of the enormous potential of the technology and the rapid pace of change. As the potential of Docker continues to evolve, Kubernetes emerges as the leading orchestration technology, and the OpenStack Magnum project has matured, many want to see shared governance over the baseline container specification and associated runtime and format/image to protect investments and enable confident adoption of this emerging technology. Join this session to learn the latest about the Open Container Initiative (www.opencontainers.org) and the Cloud Native Computing Foundation (cncf.io) - both collaborative projects of the Linux Foundation - that drive the latest cloud native technologies and projects and see how they relate to Magnum and Kuryr. Daniel Krook, Senior Software Engineer, IBM Jeffrey Borek, Program Director, Open Tech, IBM Sarah Novotny, Senior Kubernetes Community Manger, Google

Open Container Technologies and OpenStack - Sorting Through Kubernetes, the O...

Daniel Krook

Presentation at the OpenStack Summit in Austin, Texas on April 28, 2016. http://bit.ly/os-oci-cncf-ses The technology industry has been abuzz about cloud workload containerization since the open source Docker project became a phenomenon in early 2014. Meanwhile, an OpenStack Containers Team was formed and the Magnum project launched to provide users with a convenient Containers-as-a-Service solution for OpenStack environments. As the potential of both technologies emerged, many wanted to see shared governance over the baseline container specification and runtime technology to ensure an open cloud ecosystem. This past December, two new groups were launched with a goal of creating open, industry standards. The first called the Open Container Initiative (http://www.opencontainers.org), and the second called the Cloud Native Computing Foundation (http://cncf.io) Jeffrey Borek - Program Director, Open Tech, IBM - @JeffBorek Daniel Krook - Senior Software Engineer, IBM - @DanielKrook Val Bercovici - Global Cloud CTO, NetApp/SolidFire - @valb00

Containers, OCI, CNCF, Magnum, Kuryr, and You!

Daniel Krook

Building specialized container-based systems with Moby: a few use cases This talk will explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud or bare metal scenarios. We will cover Moby itself, the framework, and tooling around the project, as well as many of it’s components: LinuxKit, InfraKit, containerd, SwarmKit, Notary. Then we will present a few use cases and demos of how different companies have leveraged Moby and some of the Moby components to create their own container-based systems.

Moby Open Source Summit North America 2017

Patrick Chanezon

Opening words at DockerCon Europe by Ben Golub

Docker, Inc.

B Chapman - Codefest BOSC2012

Jan Aerts

Build your own container-based system with the Moby project Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas. Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.

Oscon 2017: Build your own container-based system with the Moby project

Patrick Chanezon

Docker EE 2.0 choice security agility by Erik Tan,Tech Insights Singapore - 2...

Ashnikbiz

See how Kubernetes-native security differs from traditional security approaches. We'll talk about how you can find and fix blind spots, critical vulnerabilities, and misconfigurations that are unique to Kubernetes to increase protection. And to get your team to adopt this, you'll also see how to help shorten the learning curve for them. Lastly, you'll see how to minimize operational risk by using scalable enforcement functions, while keeping operations simple. The demo will be on how to use Red Hat Advanced Cluster Security/Stackrox to implement Kubernetes-native security on containers that are running across k8s/OpenShift clusters and implement best practices across use cases like visibility, vulnerability management, and more. Presented by Krishnan Narayana Swamy, Specialist Solution Architect, Red Hat

[Konveyor] adding security to dev ops for your kubernetes native applications

Konveyor Community

This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment. Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm. Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.

OSCON 2014 - Crash Course in Open Source Cloud Computing

Mark Hinkle

Bosc2011 ntino-krampis-full

Bioinformatics Open Source Conference

Recording on YouTube: https://youtu.be/t0pu4iqTil0 While a lot can be modernized in the form of containers, it may not be feasible to containerize all of them at the same time due to time or technical or budget constraints. With KubeVirt, we can bring those virtualized workloads side by side with containerized workloads into a common platform. Forklift operator can bring those virtualized workloads in bulk to Kubernetes platform. In this meetup, we will show a live demo on how monolith applications can be moved, without touching the source legacy side, to Kubernetes platform. Presenter: Vishal Anand, Thought Leader Architect & Sr. Inventor - IBM

Zero-Touch Declarative Rehosting of Legacy Monolith Applications to Kubernete...

Konveyor Community

Interest in Docker has increased significantly since its inception. According to a report compiled by a leading cloud-scale monitoring company, Datadog, two-thirds of the companies that try Docker adopt it, and the adopters have increased their container count by five times over a period of nine months. Neo4j has also embraced Docker by supporting official images and also offering specific images of its own. While the interest in container technology is growing rapidly, so is the need to deploy containers over a cluster of machines to allow scalability and fault-tolerance. This highlights the need for orchestration which refers to the idea of automating the manual process of deploying, configuring and scaling the containers in an automated manner. In this talk, we provide a hands-on introduction to the three most popular Docker orchestration tools: Kubernetes, Docker Swarm and Mesos. This talk offers a conceptual understanding of each of these technologies along with an insight into the concepts learned through a series of three demos. The demos will illustrate how to deploy and automatically scale a Neo4j container using each of the three orchestration platforms. We realize that the scope of the topic in terms of the orchestration tools is too broad. The rationale behind choosing the three specific tools is based on the following two reasons: First is their potential use in our cluster at Cincinnati Children’s Hospital (CCHMC). Secondly, they also fall under the leading orchestration tools.

An Introduction to Container Organization with Docker Swarm, Kubernetes, Meso...

Neo4j

Executive Briefing: The Why, What, and Where of Containers

NVISIA

Rackspace::Solve SFO - Solve(Scale) Featuring Docker CEO Ben Golub

Rackspace

Ben keynote 5

Ben Golub

Abstract: In 2017, the industry standardized on Kubernetes as the de facto container orchestration technology. Kubernetes is an open source project, originally developed at Google, and now managed by the Cloud Native Computing Foundation. (CNCF). An orchestration engine is the basis for how operations staff deploy and manage containers at scale. In this webinar, we'll take a look at what Kubernetes as the de facto container orchestration technology means. Learn from Red Hat's leadership in the Kubernetes community and from many years of Red Hat® OpenShift customer deployments. Topics will include: • What Kubernetes delivers now. • What else is still needed to operate and maintain a container application platform. • How an application platform can optimize developer productivity, security, and multicloud deployments.

Kubernetes is the Platform. What's Next?

Brian Gracely

Presentation at the OpenStack Summit in Tokyo, Japan on October 27, 2015. http://sched.co/49x0 The technology industry has been abuzz about cloud workload containerization since the open source Docker project became a phenomenon in early 2014. Meanwhile, an OpenStack Containers Team was formed and the Magnum project launched to provide users with a convenient Containers-as-a-Service solution for OpenStack environments. As the potential of both technologies emerged, many wanted to see shared governance over the baseline container specification and runtime technology to ensure an open cloud ecosystem. This past June, a new group was formed with a goal of creating open, industry standards around container formats and runtimes, called the Open Container Initiative (http://www.opencontainers.org). So how will OpenStack Magnum influence - and be influenced by - the new OCI group? Why is the OCI under the stewardship of the Linux Foundation? What is the scope of the OCI effort? What project goals and/or principles will guide their work? Attend this session to learn the following: * A brief history of the open container ecosystem and the major benefits that containerization provides * An overview of the Magnum CaaS plugin architecture and design goals * Insider details on the the progress of the Linux Foundation Open Container Initiative (and the related Cloud Native Computing Foundation) * What it all means for deploying container orchestration engines on your cloud with OpenStack Magnum Megan Kostick - Software Engineer, Cloud and Open Source Technologies, IBM Daniel Krook - Senior Software Engineer, Cloud and Open Source Technologies, IBM Jeffrey Borek - WW Program Director, Open Technologies and Partnerships, Cloud Computing

The Containers Ecosystem, the OpenStack Magnum Project, the Open Container In...

Daniel Krook

Google has been running everything in containers for the past 15 years, but how do we orchestrate and manage all those containers? We've built and released the open source Kubernetes (http://kubernetes.io), which is based on years of running containers internally at Google. Join us for an introduction to containers and Kubernetes, followed by a hands-on workshop building and deploying your own Kubernetes cluster with multiple front end, database and caching instances. Docker containers help solve the issue of process-level reproducibility by packaging up your apps and execution environments into a number of containers. But once you have a lot of containers running, you'll need to coordinate them across a cluster of machines while keeping them healthy and making sure they can find each other. This can quickly turn into an unmanageable mess! Wouldn't it be helpful if you could declare what wanted, and then have the cluster assign the resources to get it done and to recover from failures and scale on demand? Kubernetes is here to help! Key takeaways - Gentle introduction into containers: why and how - Learn how Google manages applications using containers - Intro to Kubernetes: managing applications and services - Build and deploy your own multi-tier application using Kubernetes

Kubernetes 101 Workshop

Bret McGowen - NYC Google Developer Advocate

DockerCon EU 2017 Recap

Patrick Chanezon

Alibaba Cloud Conference 2016 - Docker Open Source

John Willis

Semelhante a BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps (20)

Open Container Technologies and OpenStack - Sorting Through Kubernetes, the O...

Containers, OCI, CNCF, Magnum, Kuryr, and You!

Moby Open Source Summit North America 2017

Opening words at DockerCon Europe by Ben Golub

B Chapman - Codefest BOSC2012

Oscon 2017: Build your own container-based system with the Moby project

Docker EE 2.0 choice security agility by Erik Tan,Tech Insights Singapore - 2...

[Konveyor] adding security to dev ops for your kubernetes native applications

OSCON 2014 - Crash Course in Open Source Cloud Computing

Bosc2011 ntino-krampis-full

Zero-Touch Declarative Rehosting of Legacy Monolith Applications to Kubernete...

An Introduction to Container Organization with Docker Swarm, Kubernetes, Meso...

Executive Briefing: The Why, What, and Where of Containers

Rackspace::Solve SFO - Solve(Scale) Featuring Docker CEO Ben Golub

Ben keynote 5

Kubernetes is the Platform. What's Next?

The Containers Ecosystem, the OpenStack Magnum Project, the Open Container In...

Kubernetes 101 Workshop

DockerCon EU 2017 Recap

Alibaba Cloud Conference 2016 - Docker Open Source

Último

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

Scaling API-first – The story of a global engineering organization

Radu Cotescu

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

Developing An App To Navigate The Roads of Brazil

V3cube

Histor y of HAM Radio presentation slide

vu2urc

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Real Time Object Detection Using Open CV

Khem

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps

1. Containeriza*on of scien*ﬁc applica*ons has done a lot to make so6ware more portable and reproducible. Unfortunately, most high performance compu*ng (HPC) clusters have been le6 out of this movement because of kernel requirements and security policy. In collabora*on with Björn Grüning represen*ng BioContainers and Vanessa Sochat from the Singularity project, we are bringing thousands of science apps to any cluster that can run Singularity. BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps John M. Fonner, Rion Dooley, Jacquelyn Turcinovic, MaWhew W. Vaughn Introduc)on Singularity Hub singularity-hub.org BioConda bioconda.github.io BioContainers biocontainers.pro Key Capabili)es •  All TACC HPC systems now support Singularity images and BioContainers •  Cyverse supports BioContainers via Docker on their cloud resources and now via Singularity on HPC resources •  Convert your own Docker containers to Singularity for use at TACC using github.com/TACC/docker2singularity Future Work •  Con*nuous integra*on from BioConda to TACC / Singularity Hub •  Enroll as a Singularity Registry on Singularity Hub •  Explore GPUs/co-processors support and custom writable directories •  On-demand conversion of Docker containers to Singularity BioConda has over 2,400 recipes for bioinforma*cs apps thanks to the contribu*ons of many researchers and developers. The Cyverse project has adopted Conda recipes as the primary method for integra*ng new apps into their infrastructure. BioContainers has automated the process of conver*ng Conda recipes into so6ware containers and has woven their framework directly into BioConda. All Docker containers are con*nuously registered with a Quay.io repository. Singularity-Hub supports integra*on with GitHub to generate Singularity images based on git commits. They are extending their code-base to support ins*tu*onal repositories of images. TACC has installed Singularity run*me support on its HPC systems, has converted all BioContainer apps to Singularity images, and has made them available to XSEDE, TACC, and Cyverse users. Public, unauthen*cated download will be available as soon as possible through Singularity Hub. Summary Get involved •  Contribute your bioinforma*cs apps to BioConda and join the community •  Try out Singularity at TACC through either xsede.org or tacc.utexas.edu •  Chime in on our development eﬀort: github.com/TACC/docker2singularity singularityhub.github.io/singularity- registry/

BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps

Semelhante a BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps (20)

Último

Último (20)

BioContainers for supercomputers: 2,000+ accessible, discoverable Singularity apps