Recent Object Detection Research & Person Detection
•
1 gostou•1,034 visualizações
Introduce recent anchor-free object detection methods on general objects and person detection. The slide summarize more than 10 papers on this topic.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Recent Object Detection Research & Person Detection
Semelhante a Recent Object Detection Research & Person Detection (20)
Mais de Kai-Wen Zhao
Mais de Kai-Wen Zhao (8)
Último
Último (20)
Recent Object Detection Research & Person Detection
- 1. Recent Object Detection Development & Person Detection Survey kv
- 2. Outline - Review Object Detection - Research Trends: Anchor-free detector - Person Detection
- 4. Object Detection ● Is deep learning dominated domain ● Modularized design, reuable ○ Components ○ Pipeline ○ Feature scaling design Object Detection in 20 Years: A Survey RCNN
- 5. General Object Detector Arch. Backbone Neck Head Backbone Neck Head Dense Head One-Stage ● YOLO ● SSD ● RetinaNet Two-Stage ● Faster-RCNN ● TunderNet
- 6. Component in Object Detection Pipeline Backbone (feature extractor) - ResNet50, ResNeXt, MobileNet - Hourglass, DLA Neck (in-net preprocessor) - RPN Dense Head - FPN, BPN, HRPN Head (task) - AnchorHead - retina, ssd - fcos, ctdet - BoxHead Loss function - CE, BCE - Focal loss - L1, Smooth L1 Computation Module - Deformable Conv (v1, v2) - GN (Group Normalization) - SyncBN - NMS, SoftNMS - GA (Guided Anchoring)
- 7. Two-Stage: Faster RCNN per ROI computation per image computation ResNet RPN Softmax RoIPool BoxReg MLP
- 8. Scale in Object Detection
- 9. Scale in Object Detection Backbone ● Without scale ○ ConvNet ● With scale ○ DLA ○ Hourglass ○ Modified-ResNet
- 10. Backbone Parameters Backbone name Top1 # of parameters FLOPs/2 ResNet-50 22.28 25,557,032 3,877.95M DLA-34 25.36 15,742,104 3,071.37M ResNet-101 21.90 44,549,160 7,597.95M Hourglass reference: https://github.com/osmr/imgclsmob/blob/master/pytorch/README.md
- 11. Person Detection
- 12. Object Detection & Person Detection Person detection ≈ class-agnostic object detection with crowdness prob.
- 13. Object Detection & Person Detection ● Crowdedness & Occlusion ● Scale & fine-grained ● Unusal pose ● Non-person, distractor ● Night scene ● Background distribution (domain shift)
- 14. Datasets COCOPerson CrowdHuman Caltech pedestrian WiderPerson WiderPerson19 CUHK Person dataset #of img #of person density COCO Person 64,115 257,252 4.01 CrowdHuman 15,000 339,565 22.64 WiderPerson 9,000 399,786 39.87 CUHK Person 18,184 99,809 5.48 WiderPerson19 sur/ad 8,240/ 88,260 58,190/ 248,993 7.05/ 2.82 Caltech pedestrian 72,782 13,674 0.32 CityPerson 2,975 19,654 6.61 train, test, benchmark
- 16. Dataset: CrowdHuman Annotations ● Full box ● Visible box ● Head box Features ● Aim Crowdness issue
- 18. Dataset: WiderPerson TMM2019 http://www.cbsr.ia.ac.cn/users/jwan/papers/TMM2019-WiderPerson.pdf Features ● Questionable annotation quality ● Limited scence distribution (by observation) Annotations ● Full box ● class, tag
- 19. Dataset: WiderPerson TMM2019 http://www.cbsr.ia.ac.cn/users/jwan/papers/TMM2019-WiderPerson.pdf Features ● More balanced location distribution
- 21. Dataset: WiderPerson2019 https://wider-challenge.org/2019.html Features ● vehicle & surveillance ● low quality but high resolution images
- 23. Observations ● Model train on COCOPerson can not perform well on real scenario (Not confirmed) ● COCOPerson contains some not reasonable annotation ● WilderPerson dataset is too noisy to use directly ● Full box is hard; visible box may cause higher fp rate ● CrowdHuman is hard but it aims to conquer crowdedness problem
- 24. Crowdedness Problem: Repulsion Loss Attraction RepGT (Repulsion Term) RepBox (Repulsion Term)
- 25. Crowdedness Problem: Repulsion Loss
- 26. Crowdedness Problem: Apative-NMS Apative-NMS ● Dynamic suppression according to target density ● Subnetwork to learn density scores
- 28. Drawbacks of anchor box ● Large #of anchors (SSD 40k, Retinanet 100k) ○ faster-rcnn low proposal still performs good ● Introduce extra hyperparameters ● May fail when mult-scale senario ● Imbalance between positive & negative anchors
- 29. Recent Trend in Object Detection
- 30. Era of anchor-free detector One-Stage: Fast, Simple Two-Stage: High Precision (Recall) Anchor-Free: Hybrid both methods 2018 - 8/3 CornerNet (pair) 2019 - 1/23 ExtremeNet (4 pts) - 4/2 FCOS - 4/8 FoveaBox - 4/18 CornerNet-Lite - 4/19 CenterNet (triplet) - 4/23 Center and Scale Prediction (CSP) - 4/25 Objects as Points (CenterNet) - 10/21 CSID (CSP+ID)
- 32. CornerNet Object as paired keypoints
- 33. CornerNet Object as a pair of keypoints (top-left & bottom-right) Find Corner Associative Embedding Grouping
- 34. CornerNet Corner Pooling Top-Left Bottom-Right Backbone matters: Hourglass provides 8 AP than FPN
- 35. CornerNet Corner Pooling Top-Left Bottom-Right ● One dimensional embedding
- 36. CornerNet: Loss function ● Pixel-wise regression on heatmap with focal loss ● Smooth L1 on offset map Heatmap OffsetGrouping
- 37. CornerNet
- 38. CenterNet: Keypoint Triplets Problem of CornerNet ● Sensitive due to edge (top 100) ● High false positive rate Improvement ● Correct prediction by checking the central parts Object as a keypoint triplet
- 39. CenterNet: Keypoint Triplets Corner Pool Associative Embedding Grouping Center Pool
- 42. FCOS Object as a point + 4d vector ● Balance between postivie & negative samples ● Ambiguous case ~ 1.4% in COCO ● Hint for center
- 43. FCOS Backbone + PFN + Head (classical arch)
- 44. FCOS: Centerness Important Feature ● Center-ness eliminates ambiguous samples ● Class score times center-ness score @NMS
- 45. FCOS: Centerness
- 46. FCOS: Improvements ● 1x and 2x mean the model is trained for 90K and 180K iterations, respectively. ● center means center sample is used in our training. ● liou means the model use linear iou loss function. (1 - iou) ● giou means the use giou loss function. (1 - giou)
- 47. Objects as Points (+2 vals) ● Simple method ○ One feature map that represents all scales ○ No bounding box matching ○ No non maximum suppression ● Better speed-accuracy trade-off
- 48. Objects as Points: “The true CenterNet” Hourglass ● Use DCNv2 instead Conv ● Heatmap supports 2D, 3D, pose estimation
- 49. Objects as Points: “The true CenterNet” ● Pixel-wise regression with focal loss ● Not normalize scale map ● Size reg. constant 0.1 ● L1 loss (rather Smooth L1) on offset loss ● Training longer performs better (140 to 230)
- 50. CSP: Center & Scale Prediction Prediction ● Center (Heatmap) ● Scale (Height) Fix aspect ratio @0.41 (according to dataset) Object as a point + 1 scalar
- 51. CSP: Center & Scale Prediction
- 52. CSP: Center & Scale Prediction Why Choose Height? Why Predict Center?
- 53. CSID: Center, Scale, Identity and Density aware ID-Map learns two measures simultaneously ● Density of predicted center ● Identity of predicted center
- 54. CSID: Center, Scale, Identity and Density aware ID-NMS
- 55. Algo Relations Anchor-Free TripletExtremeNet FCOS Single point CSP CSID Multiple points CornerNet CenterNet How points are groupped? ● Pooling ● Associative embeddings How ceneter is located? ● Centerness reg. ● Center target ● Domain contraints
- 56. Comparison Algorithm CornerNet Triplet FCOS CenterNet CSP CSID #of points 2 3 1 1 1 1, 1 Scale Backbone Backbone FPN Backbone FPN Backbone Grouping method Corner Pool Loss Center Pool Corner Pool Loss - - - ID Loss Density loss Key feature Pool Embedding Pool Centerness Simple Const. aspect ratio ID Map Post-processing NMS Soft-NMS NMS - NMS ID-NMS
- 57. Benchmarks: COCO Algorithm Backbone AP AP@0.50 AP@0.75 APs APm APl inference time YOLOv3 DarkNet-53 33 57 34.4 18.3 25.4 41.9 20 fps RetinaNet ResNeXt-101-FPN 40.8 61.1 44.1 24.1 44.2 51.2 5.4 fps CornerNet Hourglass-104 40.5 56.5 43.1 19.4 42.7 53.9 4.1 fps FCOS ResNet-101-FPN 41.5 60.7 45 24.4 44.8 51.6 - FCOS + imp ResNeXt-64x4d-101-FPN 44.7 64.1 48.4 27.6 47.5 55.6 - CenterNet DLA-34 39.2 57.1 42.8 19.9 43 51.4 28 fps CenterNet Hourglass-104 42.1 61.1 45.9 24.1 45.5 52.8 7.8 fps Centernet-Triple t Hourglass-52 41.6 59.4 44.2 22.5 43.1 54.1 3.7 fps Centernet-Triple t Hourglass-104 44.9 62.4 48.1 25.6 47.4 57.4 2.9 fps
- 58. Benchmarks: COCO Algorithm Backbone AP AP@0.50 AP@0.75 APs APm APl inference time YOLOv3 DarkNet-53 33 57 34.4 18.3 25.4 41.9 20 fps RetinaNet ResNeXt-101-FPN 40.8 61.1 44.1 24.1 44.2 51.2 5.4 fps CornerNet Hourglass-104 40.5 56.5 43.1 19.4 42.7 53.9 4.1 fps FCOS ResNet-101-FPN 41.5 60.7 45 24.4 44.8 51.6 - FCOS + imp ResNeXt-64x4d-101-FPN 44.7 64.1 48.4 27.6 47.5 55.6 - CenterNet DLA-34 39.2 57.1 42.8 19.9 43 51.4 28 fps CenterNet Hourglass-104 42.1 61.1 45.9 24.1 45.5 52.8 7.8 fps Centernet-Triple t Hourglass-52 41.6 59.4 44.2 22.5 43.1 54.1 3.7 fps Centernet-Triple t Hourglass-104 44.9 62.4 48.1 25.6 47.4 57.4 2.9 fps
- 59. Benchmarks: COCO Algorithm Backbone AP AP@0.50 AP@0.75 APs APm APl inference time YOLOv3 DarkNet-53 33 57 34.4 18.3 25.4 41.9 20 fps RetinaNet ResNeXt-101-FPN 40.8 61.1 44.1 24.1 44.2 51.2 5.4 fps CornerNet Hourglass-104 40.5 56.5 43.1 19.4 42.7 53.9 4.1 fps FCOS ResNet-101-FPN 41.5 60.7 45 24.4 44.8 51.6 - FCOS + imp ResNeXt-64x4d-101-FPN 44.7 64.1 48.4 27.6 47.5 55.6 - CenterNet DLA-34 39.2 57.1 42.8 19.9 43 51.4 28 fps CenterNet Hourglass-104 42.1 61.1 45.9 24.1 45.5 52.8 7.8 fps Centernet-Triple t Hourglass-52 41.6 59.4 44.2 22.5 43.1 54.1 3.7 fps Centernet-Triple t Hourglass-104 44.9 62.4 48.1 25.6 47.4 57.4 2.9 fps
- 60. Benchmarks: CityPerson Algorithm Name Backbone Reasonable Heavy Partial Bare inference time FRCNN VGG-16 15.4 - - - - OR-CNN VGG-16 12.8 55.7 15.3 6.7 - RepLoss ResNet-50 13.2 56.9 16.8 7.6 - CSP ResNet-50 11 49.3 10.4 7.3 3 fps Adaptive-NMS ResNet-50 10.8 54 11.4 6.2 - CSID DLA-34 8.8 46.6 8.3 5.8 6.25 fps
- 61. Training Frameworks ● Tensorflow Object Detection API ● mmdetection (CUHK) ● simpledet (TuSimple) ● Detectron, Detectron2
- 62. Conclusions ● Crowdedness is the major obstacle in person detection ● Anchor-free detector seems flexible & extensible to object task ● Center-based method + post-processing + specialized loss ○ CSID ○ CenterNet + A-NMS + RepLoss ● Trade-off between backbone & scaling level ○ ConvNet + FPN ○ DLA ● Still a challenging topic
- 63. Paper Lists: Person Detection ● CityPersons: A Diverse Dataset for Pedestrian Detection ● WiderPerson: A Diverse Dataset for Dense Pedestrian Detection in the Wild ● CrowdHuman: A Benchmark for Detecting Human in a Crowd ● CenterNet: Keypoint Triplets for Object Detection ● Objects as Points ● FoveaBox: Beyond Anchor-based Object Detector ● Feature Selective Anchor-Free Module for Single-Shot Object Detection ● FCOS: Fully Convolutional One-Stage Object Detection ● Center and Scale Prediction: A Box-free Approach for Object Detection ● Bottom-up Object Detection by Grouping Extreme and Center Points ● CSID: Center, Scale, Identity and Density-aware Pedestrian Detection in a Crowd ● Repulsion Loss: Detecting Pedestrians in a Crowd ● Adaptive NMS: Refining Pedestrian Detection in a Crowd ● Discriminative Feature Transformation for Occluded Pedestrian Detection ● PedHunter: Occlusion Robust Pedestrian Detector in Crowded Scenes ● Occlusion-aware R-CNN: Detecting Pedestrians in a Crowd ● Double Anchor R-CNN for Human Detection in a Crowd