This document summarizes a presentation about using Lightning Web Components (LWC) to build a datatable to search for products from a large dataset. It discusses challenges faced, such as platform restrictions, performance issues, and type coercion errors when trying to query fields. It provides tips for handling large data volumes, like using offsets, limits, and Apex for sorting. While promising, LWC datables have limitations and require special handling of features like selections and relationship fields. Open source tools can help generate test data to explore solutions before projects launch.
Having serious fun with LWC Datatable and Large Data Volume
1. Having serious fun with LWC Datatable
and Large Data Volume
by Christian Szandor Knapp
& Christian Menzinger
2. #CD22
About us
Christian Menzinger
Salesforce Architect, Copado
@chris_menzinger
https://www.linkedin.com/in/christianmenzinger
Christian Szandor Knapp
Head of Development, appero
@ch_sz_knapp
https://www.linkedin.com/in/szandor
3. #CD22
- Chapter I - In a Land far, far away
- Chapter II - Trails for Rangers, Crossroads for Heroes
- Chapter III - The mole’s tale - Summer is coming
- Chapter IV - The dwarf’s tale - We dig, dig, dig, dig, dig, dig, dig
- Running out of time
- Q&A
Your Fairytale Today
9. #CD22
We have a need, she said.
It is going to be beautiful, she
said.
It will be so much fun to build,
she said.
We use base components,
she said.
Until one day …
10. #CD22
A meeting, two days before …
His dog,
Franz K.,
Architect
Herr Governor Limit
Karl,
4th Gen CEO
Inherited an Empire
(Pricebook)
13. #CD22
Franz, the Architect (sometimes a little too busy)
We always empower Admins.
Have them configure a List View on
Product2.
Use the List View to define the table.
By the way, a Pricebook has 1.500.000
products per currency. Could get a lot
more very quickly.
14. #CD22
We have a long standing SAP business relationship.
Picklist Values/Keys on Product2 are internal to SAP
Picklist Labels give User Guidance
Use localized Picklist Labels
Make sure fields like Term (12, 24, 36) can be
searched.
Karl, the Boss (Trailhead Ranger, Part Time Admin)
19. #CD22
Front End / LEX
(Base Components)
Backend
(Apex)
Front End / LEX
(Wire Adapters)
Project Setup
(e.g. Test Data)
Where to next? Agile or Waterfall?
Backend
(sObjects)
21. #CD22
Official cause of death
Governor Limit
RI
P
You died
Invalid Type
Coercion
Platform
Restrictions
Fuzzy (Ideas about)
Requirements
Racing Conditions /
Flags
Performance Issues
currently in Beta
or Deprecated
26. #CD22
He’s a curious and crafty krtek
I’m off building a tiny Proof of Concept
I’m back in an hour
27. #CD22
He’s a curious and crafty krtek
I will
- Set up a Scratch Org with sample
data using VS Code
- Pump it full of products
- Create LWC
- Use LWC Local Development for
fast prototyping
- Use Wire Adapter to load list view
and records! I used that last year
28. #CD22
Official cause of death
RI
P
You died
Platform
Restrictions
Racing Conditions /
lags
currently in Beta
or Deprecated
29. #CD22
Official cause of death
RI
P
You died again
Platform
Restrictions
Racing Conditions /
lags
currently in Beta
or Deprecated
Needs a higher API version than
supported by LWC Local Development
35. #CD22
Here’s what we’ve learned
- A lot about client side performance
- Why Summer 22 will make it better
- Why Selections need special care
- Why there are no shortcut/s via Listview
APIs
- Decisions
- No data, no table, no columns, no table
- To talk to Apex we need to know which fields to query
first
- Use getListInfoByName()
- Salesforce Fields are also columns
- Columns need type information
- E.g. Booleans should render as a checkboxes
- Use getObjectInfo()
36. #CD22
Although we called it
“Tabular Product
Search”
It is not Products we are
looking for but
PricebookEntries
Official cause of death
RI
P
You died
Fuzzy (Ideas about)
Requirements
37. #CD22
Official cause of death
RI
P
You died again
Platform
Restrictions
PricebookEntries
Do
Not
Support
ListViews
38. #CD22
Official cause of death
RI
P
You almost died
a 3rd time
Racing Conditions /
Flags
Wire Adapters
are supported only in
@wired decorated
properties/function
No Promise.all()
available
39. #CD22
Official cause of death
Governor Limit
RI
P
You died
a 3rd time
Performance Issues
currently in Beta
or Deprecated
40. #CD22
Here’s what we’ve learned
- Loading fewer records is always better
for performance
- Use infinite loading
- Use offsets to avoid query limits for
LDV
- Use a record limit / threshold for
rendered rows in table
- Have we talked about sorting yet?
- String.prototype.localeCompare()
- Or Apex?
44. #CD22
Here’s what we’ve learned
- Loading fewer records is always better
for performance
- Use infinite loading
- Use offsets to avoid query limits
- Use a record limit / threshold for
rendered rows in table
- Apex does the sorting
- Works good within caching
- Needs a bit of loading otherwise
45. #CD22
Summary I
- LWC provides
- Offsets
- Limits
- Search Term
- SortedBy Field and Sort Direction
- LWC is responsible for
- Providing Initial Data for Apex Query
- e.g. Field Names
- Talking to Apex
- Debouncing
- Selection Handling
- Data Wrangling
- Data Table does not support
Contact.Account.Name
54. #CD22
Summary II
- Dig, Dig, Dig, Dig, Dig, Dig, Dig
- Not all field types can / will be supported for text search
- (Idea) specialised search fields for e.g. date
- All supported types need to be converted for queries
- Product2.Description is a special snowflake
- (Idea) SOSL for long text fields
- (Challenge) combine User Experience
55. #CD22
Summary II
- Will we meet Large Data Volume Trouble?
- No due to tight LIMIT clauses
- Rules differ for standard and custom indices
Standard Index
Total # of records is < 1M:
Query must return less than 30% of
total
Total # of records > 1M:
Query must return <300K rows
Custom Index
10% of total records and < 10k rows
57. #CD22
Architectural Mindset: keep asking questions
Challenge “simple-use-case” documentation
Commit early, commit cleanly, commit often
Handling events as they come out of the box is not always the whole truth
Product2, PricebookEntry do not count against record limit
Inserting 1.5m records is easy, LDV Selective Query Limits are not.
Huge Gains in Summer 22 for Datatable Render Performance
Random Collection of Take-Aways
“What did we miss in the meantime?” “ A meeting, two days before between Franz and Karl …. “
Why not Pricebook Entry List View? Because not available / supported . admins cannot configure listviews on PBE
Answer: Partly, yes. Opp Product Search works only in one place, though. We build a level more generic - the same search can be used for opp products as well as order products for example. Assets could work similarly.
See, if our heroes had only known what expected them.
But they didn’t. Ignorance is a bliss sometimes.
Hop into documentation
Scratch Org timeouts didn’t help, either…
Krtek made his demo much simpler;
Now, how can that happen in the very first PoC?
NO death by governor limit?
NO death by governor limit? Nope; not even data limits :)
It is still Spring, virtual rendering is only available in Sandboxes / Preview Orgs
Even if we could why would we want to load 5000+ records?
Given we have a 1.5 M Products (and pricebook entries), we could hit the 50.000 query row limit
The most records that could ever be returned for the query to be considered selective is 1M but the total data set size would need to be > 5.6M records
Steve Baines
The most records that could ever be returned for the query to be considered selective is 1M but the total data set size would need to be > 5.6M records
Steve Baines
I have a question: why was there never any mention of large data volume again?
I have a question: why was there never any mention of large data volume again?