In 2022, I launched a Kotlin podcast show, Kotlin Fireside Chat, with my friends in Kotlin User Group. Now, we're taking things to the next level by building a podcast website with a statistical analysis dashboard. We've built a static website using Jamstack and connected it to a serverless API written in Kotlin. In order to aggregate the listener number across platforms, such as YouTube, BiliBili, Ximalaya, Lizhi, Qingting, we've also created a crawler to grab those numbers and display them in a custom dashboard. Everything is made by open source technology and deployed on Google Cloud Platform. In this talk, I'll share my experience and the lessons I learned when using all the technologies in 2023.
11. An evolution process
of the show
—
• Make a logo, pick a theme music and start!
• Prepare a microphone for better sound quality
• Start being picky with background and lighting
• Put more efforts on post editing and design
• Working on SPECIAL ROADSIDE episode recently
16. Overall structure
—
• Website - a bunch of static webpages
• Data API - provide content to the website
• Crawler - grab number from different platforms
• Report - give an image of the trend
17. Technology selection
—
• Website - Jamstack approach with Astro + Cloud Storage
• Data API - Cloud Functions with Kotlin
• Crawler - jsoup with Kotlin + Cloud Functions & Scheduler
• Report - Spring Boot with Kotlin + Vaadin Chart
22. Create
MainLayout
—
---
export interface Props {
title: string;
}
const {title} = Astro.props;
---
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<title>{title}</title>
</head>
<body>
<slot/>
</body>
</html>
pass page-specific values as props
#
Scripts for this component
#
HTML content for a layout
"
"
use a slot element to place page contents within a layout
23. Use layout
on a page
—
---
import MainLayout from "../layouts/MainLayout.astro";
---
<MainLayout title="⾸⾴">
<main id="content" class="padding-top-bottom">
<div class="container">
</div>
</main>
</MainLayout>
<style>
</style>
#
Import the component
#
Use MainLayout component
#
Apply style only to this page
24. Make reusable
components
—
---
---
<header id="top" class="navbar navbar-sticky">
// ...
</header>
---
import Header from "../components/Header.astro";
---
<body>
<TopHeader!"
<slot !"
!#body>
#
Header component
#
Any other components " put component in a page
" import
25. Make a home using
Cloud Storage
—
• Create a Bucket using domain name on Cloud Storage
• Verify domain name ownership via Google Search Console
• Point the domain name to Cloud Storage
• Make Bucket accessible publicly
• Specifies the object name for default page
• Specifies the error page (404) to serve
30. Data Class
—
@Serializable
data class Episode(
val id: Int,
val slug: String,
val title: String,
val abstract: String,
val content: String,
val timeline: String,
val cover: String,
val permalink: String,
val publishedAt: String,
)
31. class Handler: HttpFunction {
@Throws(IOException::class)
override fun service(req: HttpRequest, res: HttpResponse) {
// ...
}
}
HTTP Function
—
#
Extend from HttpFunction
#
Implement service()
32. class Handler: HttpFunction {
@Throws(IOException::class)
override fun service(req: HttpRequest, res: HttpResponse) {
// ...
val data = Json.encodeToString(
mapOf("data" to episodes)
)
with(res) {
setStatusCode(HttpURLConnection.HTTP_OK)
setContentType("application/json")
writer.write(data)
}
}
}
JSON response
—
#
Setup response details
#
Serialization collection
33. CI/CD using
Cloud Build
— steps:
- id: 'Build'
name: 'gcr.io/cloud-builders/mvn'
args:
- 'clean'
- 'verify'
- id: 'Deploy'
name: 'gcr.io/cloud-builders/gcloud'
args:
- 'functions'
- 'deploy'
- '...'
- '--trigger-http'
- '--allow-unauthenticated'
- '--region'
- '...'
- '--gen2'
- '--runtime'
- 'java17'
- '--memory'
- '...MB'
- '--entry-point'
- '...'
#
Build project using maven
#
Deploy using gcloud
Choose memory size
"
Specify the entry class name
"
Choose JVM runtime
"
Use 2nd Gen Functions
"
Choose region
"
Open for HTTP without auth
"
36. Loading data in
Astro
—
---
let response = await fetch("...")
let episodes = await response.json();
---
<div class="episodes-listing">
{
episodes.map((episode) => (
<Episode slug={`${episode.slug}`}
title={`${episode.title}`}
cover={`${episode.cover}`}
abstract={`${episode.abstract}`}/>
))
}
</div>
#
API call to get data
#
Listing data in a component
37. Generate pages in
Astro
— // [id].astro
---
const { id } = Astro.params;
let response = await fetch(`.../${id}`)
let episode = await response.json();
export async function getStaticPaths() {
let data = await fetch("...")
let episodes = await data.json();
return episodes.map((episode) => ({
params: { id: episode.id },
props: { episode: episode },
}));
}
---
<header id="..." class="...">
<h1 class="..." set:html={episode.title}/>
</header>
#
Fetch data for single page
#
getStaticPaths()
should return an array
of objects to determine
which paths will be pre-
rendered
#
Display the episode data
uses dynamic params in pages’ filename
"
39. jsoup:
a HTML Parser
—
https://jsoup.org/
// fetch the webpage
val url = "..."
val doc = Jsoup.connect(url).get()
// use selector-syntax to find elements
val elements = doc.select("div.info div.category .count")
// extract what you need from elements
elements.text()
elements.html()
elements.outerHtml()
elements.attr("...")
40. Grab text from
喜⻢拉雅
— <!-- webpage html -->
<div class="info kn_">
<div class="category kn_">
<span class="count kn_">
<i class="xuicon xuicon-erji1 kn_"></i>
546
</span>
</div>
</div>
// fetch webpage
val url = "https://www.ximalaya.com/sound/$soundId"
val doc = Jsoup.connect(url).get()
val element = doc.select("div.info div.category .count")[1]
// grab text
element.text().trim().toInt()
41. Grab text from
YouTube
— <!-- webpage html -->
<!DOCTYPE html>
<html>
<head>
<div>
<meta itemprop="interactionCount" content="128">
</div>
</head>
</html>
// fetch webpage
val url = "https://www.youtube.com/watch?v=$videoId"
val doc = Jsoup.connect(url).get()
val element = doc
.select("meta[itemprop=interactionCount]").first()
// grab text
element?.attr("content")?.toIntOrNull() ?:0
52. Technologies
—
• Astro for website
• Function Framework with Kotlin for API
• jsoup with Kotlin for crawler
• Spring Boot + Vaadin Chart for Report
54. Moving forward
—
• Paying down technical debt
• Support 2 locales on website
• Download/Import CSV data from Google, Apple, Firstory
• Build cross-platform mobile app
57. How I make a podcast website
Photo by Michal Czyz on Unsplash
using serverless technology in 2023
Shengyou Fan (范聖佑)
JetBrains Developer Advocate
shengyou.fan@jetbrains.com
Thank you!