O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a
Creative Commons Attr...
Próximos SlideShares
Carregando em…5
×

Spring For Apache Kafka

3.994 visualizações

Publicada em

SpringOne Platform 2016
Speaker: Gary Russell; Principal Engineer, Pivotal

With the advent of the Kafka 0.9+ pure Java client, the Spring Team has created the new spring-kafka project with familiar Spring abstractions such as the KafkaTemplate, message listener container and POJO listener annotation @KafkaListener.

In this talk we'll take a look at the features of the project as well as the new version (2.0) of spring-integration-kafka which is now based on the Spring for Apache Kafka project.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Spring For Apache Kafka

  1. 1. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring for Apache Kafka Gary P. Russell @gprussell
  2. 2. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring for Apache Kafka • Brings the familiar Spring programming model to Apache Kafka • KafkaTemplate • MessageListenerContainer • @KafkaListener Annotation http://projects.spring.io/spring-kafka/ 2
  3. 3. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring for Apache Kafka spring-kafka: • 1.0.2.RELEASE (0.9.x.x client) • 1.1.0.BUILD-SNAPSHOT (0.10.x.x client) - milestone soon • https://github.com/spring-projects/spring-kafka spring-integration-kafka: • inbound/outbound channel adapters • 2.0.1.RELEASE (based on spring-kafka) • 1.3.1.RELEASE (uses 0.8.x.x client directly) • https://github.com/spring-projects/spring-integration-kafka 3
  4. 4. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ProducerFactory 4 @Bean public ProducerFactory<String, String> producerFactory() { Map<String, Object> props = new HashMap<>(); props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, this.brokerAddress); // ... props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class); return new DefaultKafkaProducerFactory<>(props); }
  5. 5. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ KafkaTemplate 5 @Bean public KafkaTemplate<String, String> kafkaTemplate() { return new KafkaTemplate<>(producerFactory()); } ListenableFuture<SendResult<K, V>> send(String topic, V data); ListenableFuture<SendResult<K, V>> send(String topic, int partition, V data); etc.
  6. 6. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ConsumerFactory 6 @Bean public ConsumerFactory<String, String> consumerFactory() { Map<String, Object> props = new HashMap<>(); props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, this.brokerAddress); props.put(ConsumerConfig.GROUP_ID_CONFIG, "s1pGroup"); // ... props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); return new DefaultKafkaConsumerFactory<>(props); }
  7. 7. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ @KafkaListener 7 @KafkaListener(id = "foo", topics = “${my.topic}") public void listen1(String foo) { // ... } @KafkaListener(id = "bar", topicPattern = “${topic.root}.*”) public void listen2(@Payload String foo, @Header(KafkaHeaders.RECEIVED_MESSAGE_KEY) Integer key, @Header(KafkaHeaders.RECEIVED_PARTITION_ID) int partition, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) { // ... }
  8. 8. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ @KafkaListener - Explicit Partition Assignment 8 @KafkaListener(topicPartitions = @TopicPartition(topic = "${kafka.topic}", partitionOffsets = { @PartitionOffset(partition = "0", initialOffset = "0"), @PartitionOffset(partition = "1", initialOffset = "0"), })) public void listen(String foo) { // ... }
  9. 9. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ @KafkaListener - Container Factory 9 @Bean public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory() { ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>(); factory.setConsumerFactory(consumerFactory()); factory.setMessageConverter( new StringJsonMessageConverter()); return factory; }
  10. 10. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Listener Containers 10 @Bean public KafkaMessageListenerContainer<String, String> container( ConsumerFactory<String, String> consumerFactory, ConfigProperties config) { ContainerProperties containerProperties = new ContainerProperties(config.getTopic()); containerProperties.setMessageListener(listener()); return new KafkaMessageListenerContainer<>(consumerFactory, containerProperties); }
  11. 11. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ MessageListener 11 public class Listener implements MessageListener<String, String> { @Override public void onMessage(ConsumerRecord<String, String> record) { // ... } }
  12. 12. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ AcknowledgingMessageListener 12 public static class AckListener implements AcknowledgingMessageListener<String, String> { @Override public void onMessage(ConsumerRecord<String, String> record, Acknowledgment acknowledgment) { // ... acknowledgment.acknowledge(); } }
  13. 13. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Listener Adapters • Add behavior to listeners • Retry adapter • Filtering adapter • Also configurable when using @KafkaListener 13
  14. 14. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring for Apache Kafka Test Support • Embedded Kafka Server - JUnit @Rule • Creates topics with configurable partitions • Utility methods to wait for partition assignment • Hamcrest Matchers • AssertJ Conditions 14
  15. 15. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Some Caveats • Polling, Pause, Resume, auto commit problems • Let the container manage the commits (or manual) 15
  16. 16. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring for Apache Kafka DEMO 16
  17. 17. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Learn More. Stay Connected. Contribute!! https://github.com/spring-projects/spring-kafka/CONTRIBUTING.adoc @springcentral spring.io/blog @pivotal pivotal.io/blog @pivotalcf http://engineering.pivotal.io

×