2. NETRA: Interactive Display for Estimating Refractive Errors and Focal Range Vitor Pamplona Ankit Mohan Manuel M. Oliveira RameshRaskar 2
3. 3 Millions have poor vision, but are not getting corrected… Kenya 2B have refractive errors 0.6B have URE 4.5B have a cell phone India 6.5 Billion people 3
16. Perfect Vision System Infinity Subject can focus at infinity Human Eye Accommodation Range Normal Vision 10cm Infinity 16
17. Myopia (nearsightedness) Infinity Subject cannot focus at far distances Wrong focal point Human Eye Accommodation Range Normal Vision Myopia 10cm Infinity 17
18. Myopia Correction Infinity Subject can focus at infinity Divergent Lens Human Eye Accommodation Range Normal Vision Corrected Myopia Myopia 10cm Infinity 18
24. Relaxed Eye with Myopia Eye Red pointat infinity Blurred point Focusing Range perfect vision myopia hyperopia ~10cm infinity 24
25. Relaxed Eye with Myopia Eye Pinholes Distinct image points Red pointat infinity Focusing Range perfect vision Scheiner’s Principle myopia hyperopia ~10cm infinity 25
26. Relaxed Eye with Myopia Eye Display A Distinct image points Virtual red pointat infinity B Focusing Range perfect vision myopia hyperopia ~10cm infinity 26
27. Relaxed Eye with Myopia Eye Display Move spots towardseach other A Distinct image points Virtual red pointat finite distance B Focusing Range perfect vision myopia hyperopia ~10cm infinity 27
28. Relaxed Eye with Myopia Eye Display Move spots towardseach other A Points overlap Virtual red pointat finite distance B Focusing Range perfect vision myopia hyperopia ~10cm infinity 28
29. Relaxed Eye with Myopia Eye Display Move spots towardseach other A Points overlap Virtual red pointat finite distance B Focusing Range perfect vision myopia hyperopia ~10cm infinity 29
30. Relaxed Eye with Myopia Eye Points overlap Point at infinity Focusing Range perfect vision myopia hyperopia ~10cm infinity 30
31. Relaxed Perfect Eye Display A Points overlap Virtual red pointat infinity B Focusing Range perfect vision myopia hyperopia ~10cm infinity 31
32. Relaxed Eye with Hyperopia 32 Eye Display A Distinct image points Virtual red pointat infinity B Focusing Range perfect vision myopia hyperopia ~10cm infinity
33. Relaxed Eye with Hyperopia Move spots awayfrom each other Display Display A Points overlap B Virtual point“beyond” infinity Focusing Range perfect vision myopia hyperopia ~10cm infinity 33
34. Relaxed Eye with Hyperopia Move spots awayfrom each other Points overlap Virtual point“beyond” infinity Focusing Range perfect vision myopia hyperopia ~10cm infinity 34
60. Measuring the Accommodation Range 52 Myopia Perfect vision Hyperopia ~10cm Infinity Step 2: Near limit Step 1: Far limit
61. Measuring the Accommodation Range 53 Myopia Perfect vision Hyperopia ~10cm Infinity Step 2: Near limit Step 1: Far limit
62. Measuring the Accommodation Range 54 Myopia Perfect vision Hyperopia ~10cm Infinity Step 2: Near limit Step 1: Far limit
63. Relaxed Eye Display A Points overlap Virtual Point at the far limit B 55
64. Accommodated Eye Display Move points towards each other A Points overlap B 56 Virtual pointgetting closer Subject Accommodates to fix the “blur”
65. Accommodated Eye Display Move points towards each other A Points overlap B 57 Virtual pointgetting closer Subject Accommodates to fix the “blur”
66. Accommodated Eye Display Move points towards each other A Points overlap B 58 Virtual pointgetting closer Subject cannot accommodate more than the previous point
67. Patterns for Alignment Task 59 A B A B A B A B A B Displayed Subject view A B A B A B A B A B Displayed Subject view Visual Cryptography [NaorShamir94]
68. Patterns for Alignment Task 60 A B A B A B A B A B Displayed Subject view A B A B A B A B A B Displayed Subject view Visual Cryptography [NaorShamir94]
69. Summary of Interaction Accommodation Range Farthest Point (myopia, hyperopia, astigmatism) NearestPoint (presbyopia) 61
70. Accuracy Sharpness Estimation is subjective Brightness affects results Pupil size variation and DoF Cost Trial Lens Set > $150 Bulky Snellen chart Phoropter Trial lenses Reading Charts
72. Limitations Children Ability to align lines Single Eye test Other eye for convergence-forced accommodation Resolution is a function of the display DPI Samsung Behold II – 160 DPI – 0.35D Google Nexus One – 250 DPI – 0.2D Apple iPhone 4G – 326 DPI – 0.14D 64
73. Media Coverage BBC CNN NBC MIT News O Estado de SP - Brazil Gizmodo NY Times Time - Wellness
80. CATRA: Cataract Screening Tool Unique, quantitative lens mapping for size and density of eye opacities
81. Testing the Presence of Cataracts 73 Blinking patterns on Screen Pinhole Lens Cell Phone Display
82. Point Spread Function Mapping 74 Blinking patterns on Screen Pinhole Lens Cell Phone Display
83. Point Spread Function Mapping 75 Blinking patterns on Screen Pinhole Lens Cell Phone Display
84. Traditional User Driven Mass-use Devices Scientific Instruments
85.
86. For the Future Ophtalmology for Masses >.5 billion URE. > 2.5 billion RE. -> Devices for all Quality of phones (resolution) will increase exponentially New features (recently cataracts, next Retinal Netra) Smart phones will take over the market in developing world countries like India in next 5 years. Hardware store
87. New Ecosystem Medical IT Systems EHR Eyecare Providers Delivery Diagnostic Asynchronous
Not just academic curiosity but potential for large impactWe call our tool NETRA: near eye tool for refractive assessmentsuch as nearsightedness/far/astigmatismBasic idea is to create a unique interactive lightfield display near the eye and is possible due to the highresolution of modern LCDs.
In this paper, we show a self-optometry solution. You look at a cell phone display thru a clip-on eye piece, interactively align a few patterns, hit calculate and get data for your eye prescription.
2 billion people have refractive errorsAnd half a billion in developing countries worldwide have uncorrected vision that affects their daily livelihood. They don’t have access to an optometrist or it simply too expensive. While making and distributing of lenses has become quite easy now, surprisingly there isstill no easy solution for measuring eyesight.Can we use a fraction of the 4.5B cellphone displays to address this problem?
In 1960s, photography equipment was really crappy. They were expensive and bulky equipment, require specialized training, with high maintenance costs and they were not smart at all. But the worse thing about photography in that time is that you must go to a specific place to take the picture and then go back to get the results.
Well, today things changed. Each one of us carries at least 3 cameras: two eyes and a cell phone camera. Cameras are everywhere. They became cheap, accessible and easy-to-use without losing in accuracy.
Now, if you think about optometry today, the devices are expensive and bulky, they require specialized training, have high maintenance costs and they are not smart at all. Some of them do not even communicate with facebook. But the worse thing is that you must go to a specific place to take the eye exam and then go back to get the results.
So, we propose the increase of accessibility for optometry solutions by using high end scientific devices: cell phones. An smartphone screen today has the pixel size of 30 micrometers. At this resolution, the smartphone is not a phone anymore it is a scientific tool. With 4.5 billion phones out there, we can scale optometry and find half a billion people that today do not know that they need glasses.
The most accurate method is based on a so called SH WS. It involves shining a laser at the back of the retina and observing the wavefront using a sophisticated sensor.We ask user to generate a spot diagram. But navigating in a high dimensional space ischallenging so we come up with a strikingly simple approach to let the user interactively create the spotdiagram.We are first to make connection between Shack Hartmann and Lightfields (and it goes well with recentwork in computational photography about ALF and Zhang/Levoy). Connection to Adaptive optics/Astronomy. The way that this device works is that, it shines a lasers in the eye, the laser is reflected in the retina and comes out of the eye being distorted by the cornea. These light rays reaches an array of lenses that focus them to dots in a sensor. The device measures how much this dots deviate from the ideal case. Since it uses lasers, the device is expensive and requires trained professionals
For a normal eye, the light coming out of the eye forms a parallel wavefront. The sensor has a lenslet array and we get a spot diagram of uniform dots.This lenslet should remind you of a lightfield camera, and in fact Levoy and others showed last year that there is a close relationship between the two.In addition, Zhang and Levoy, plus our grp has shown the relationship between wavefront sensing and lightfield sensing.
When the eye has a distortion, the spot diagram is not uniform.And the displacement of the spots from the center indicates the local slope of the wavefront. From the slope one can integrate and recover the wave shape.
NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user.And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently.But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
For eye with distortion, the user will interactively displace the 25 points so that he will see a single spot. Of course changing 25 spot locations is cumbersome, but we realize that there are only 3 parameters for eye-prescription and we help the user navigate thru this space efficiently.But if you think about these theory, you will realize that we have the dual of the shack-hartmann. First we though out the laser.
The human eye is like a camera. It has lenses, sensors and also aberrations. The human eye is composed of two main lenses: the cornea, which is main responsible for converging light rays to the retina; and the crystalline lenses, which is responsible for our ability of focus far and close by changing its shape.
So, in a perfect vision system, the light coming from a point at infinity will converge to a single point at the retina. A subject with perfect vision see clearly from infinity to up to 10cm.
Myopes cannot see far. Therefore, all the rays coming from a point at infinity, converges before the retina. The Accommodation range for those people is shifted to close, so they can closer than regular individuals.
The correction for myopia includes a divergent lens, which brings the focal point back to the retina by shifting the Accommodation range.
Hyperopes cannot see close. All the rays coming from a point at infinity, converges behind the retina. The Accommodation range for those people is shifted to the far field, so they can actually see “beyond infinity”. This remembers-me some other story, but let keep the focus here.
The correction for myopia includes a convergent lens, which shifts the Accommodation range back to the regular indivudial.
The correction for myopia includes a convergent lens, which shifts the Accommodation range back to the regular indivudial.
We need to measure the difference between the subject’s farthest focal point wrt infinity.
And this is measured in diopters which is 1 divided by this distance.
So, lets start with an eye with myopia. Remember, they cannot see far, so a red point at infinity for them will look like a red blur.
Using Shceiner’s principle, if we put two pinholes in the field, this will instead create two distinct dots.
Instead of a distant point source, we put an LCD display behind the pinholes. If we draw two spots exactly under these pin-holes, we create a virtual point at infinity.
So, as we move the two red circles toward each other, the virtual point gets closer to the subject and he sees the two red dots getting closer.
When this two red circles overlaps for the subject, we can compute d based on the spot displacements
Which is the distance between the eye and this virtual point.
Turns out that the inverse of D is the refractive power required for this person to see clearly objects at infinity. In other words, the lens that will shift the accommodation range of this subject back to the regular one.
In case of a perfect eye using the system, since the subject can see far, he will see the two points overlapping in his retina, meaning that he does not need glasses.
Hyperopes focal point is behind the retina.
When they move these spots away from each other, we are moving the virtual point beyond infinityAnd buzz lightyear will entually see they overlap, and when this happens, we can compute the…
convergent lens required to shift their accommodation range to the normal stage.
The version that I showed to you uses pinholes to encode the apperture.
However, if we change these pinholes for lenses, we can increase the light and also the number of testing points in the corneal surface, meaning that we can actually create a map of one’s refractive error. As you can see the pixel pitch directly affects the precision of creating virtual depth as well as refraction estimation.
And number of clicks required for alignment indicates the refractive error
In practice we display lines on the screen and the subject overlaps these lines by pressing the buttons of the cell phone or in the computer.
Two main benefitsNo moving partsBlur into a more objective alignment problemUnfortunately, the lightfield and virtual point analogy does not extend to astigmatism and we can also compute ‘focal range’ rather than just relaxed state. Vitor will cover this.”ThanksRamesh, There is a third condition called astigmatism
which is anangle-dependent refractive error. An astigmatic subject has two main focal lengths in perpendicular meridians. One …
Stronger and one weaker
Think of a cornea with the shape of an american football creating a cylindrical aberration with unknown focal length and axis.
The required correction is now a function of measured angle. In order to measure the farthest point for these guys, we need to evaluate Cylindrical component, the Spherical component, and the angle theta on the equation. However, the interpolation of refractive powers between C and S leads to a situation where the pattern drawn on the screen matters.
As you can see in this video, the astigmatic lenses create a deviation on the path of the pattern, and they may never overlap, turning the alignment task into a 2D search for some angles.
However, if we drawn lines perpendicular to the measured angle, the alignment task is again an 1D search. The deviation still exists, but the pattern makes the task easier.
So, we do the alignment task for a few meridians
By showing oriented lines on the display.
In the end, we best fit the sinusoidal curve over the four measured values to estimate the astigmatic parameters.
In the end, we best fit the sinusoidal curve over the four measured values to estimate the astigmatic parameters.
Ours is the only system where one can estimate not only the farthest point
one can focus but also
the nearest point without any mechanically moving parts. So, in order to measure the closest reading point
We draw a pattern on the screen that induces accommodation. In this way, when we move A and B closer on the screen,
the user will try to focus on a closer object. We can move this virtual point all the way to the nearest discernable point.
When the user is not able to focus anymore, the visual system give up and the user start seeing more than one pattern.
As I sad before, this is possible because we can draw whatever we want in the display. We tested many patterns, static and dynamic, including visual cryptography.
Turns out that the best pattern to induce accommodation is the sinosoidal curves aligned perpendicular to the measurement angle.
As a summary, our method has two steps. First measures the farthest point in focus in many angles using lines and the second step measures the nearest point using sinusoidals oriented on the angle of astigmatism.
Reading charts appear to be an easy solution, this method has too many problems. Sharpness of legible text is very subjective. The brightness of the chart has to be very carefully chosen otherwise the pupil size will change, increasing depth of field, and allowing user to recognize even lower rows.The trial lenses + the lens frame the doctor will use also cost over $150% Reading chart tests involve using a frame or a phoropter. The doctor will swing a sequence of lenses in front of your eye and ask for which lens allows you to see the lower rows on the reading chart.
For better precision, there are many kinds of solutions, some really clever. The beauty of netra is that it avoids moving parts or shining lasers, and all intelligence is in the software.
Since we are relying on the user interaction, the subject has to be aware of the alignment tasks. So, very young Children may not be able to run the test. Instead of just one eye, one may use both eyes to exploit convergence. And of course, the resolution of NETRA itself is a function of the resolution of the display. With a 326 dpi display, resolution is 0.14 diopters and presciption glasses come in increments of 0.25 diopters. So our system is already sufficiently accurate.
NETRA matches Retinoscopy
About 8 million worldwide are blind (worse than 3/60 vision) because of uncorrected refractive error, mostly from the developing world…3million from India. About 22.5 million worldwide are blind because of cataracts, 19 million in the developing world, 14 million in India. From lowered productivity to less independence in conducting simple tasks, the burden of blindness is well known. This can be solved if they had access to a diganostic test and glasses, but they don’t have access to an optometrist or it simply too expensive.While making and distributing of lenses has become quite easy now, surprisingly there is still no easy solution for measuring eyesight.Can we use a fraction of the 4.5B cellphone displays to address this problem?
NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user.And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user.And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
NETRA uses an exact inverse of this sensor. We get rid of the laser and we instead show the same spot diagram in a cellphone display. For normal eye, it will appear as a dot to the user.And then we replace the sensor for a light field display. If the user sees a single red dot, he does not need glasses, but if he sees more than one, he interacts with this display.
Power for user intelligence can overcome very cumbersome and expensive devices. But unlike other condition eye screening is quite challenging.Modern solutions may provide students a fighting charge is a very rewarding.
New wireless eyecare ecosystemAnybody can take netra to patients, load .. Mobile partners, Deliver ..Because it is mobile and deskilled, breaks the barrier to entry, takes eyecare to remote areasDecouple diagnostics from delivery
Thanks XXXNETRA is a clip-on device that you attach to your cell phone. You look close, press some buttons, you hit calculate and it gives you the prescription for glasses. It’s a 2-dollar device that measures nearsightedness, farsightedness and astigmatism with the same accuracy that doctors have in their clinic.To understand what happened here, let’s think about the evolution of photography.
