O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
What is computer-aided summarisation and does it really work? Constantin Orasan http://clg.wlv.ac.uk/projects/CAST/
Structure <ul><li>Introduction </li></ul><ul><li>CAST </li></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions </li></ul>
Computer-aided summarisation <ul><li>Combines automatic methods with human input </li></ul><ul><li>Relies on automatic met...
<ul><li>Automatic summarisation (AS) </li></ul><ul><li>Produces summaries automatically with the help of computers </li></...
Why CAS can work? <ul><li>Endres-Niggemeyer (1998) identifies three stages in human summarisation:  document exploration ,...
Computer-aided summarisation tool (CAST) <ul><li>Work funded by Arts and Humanities Research Council </li></ul><ul><li>Wor...
CAST- the tool (II) <ul><li>At present CAST contains the following methods: </li></ul><ul><ul><li>Keyword method </li></ul...
 
 
 
Feedback from the user <ul><li>We analysed the work of our human summariser </li></ul><ul><ul><li>Term-based summarisation...
Evaluation <ul><li>Our assumption about CAS is that it is possible to produce summaries in less time without any loss in q...
Experiment 1 <ul><li>Used one professional summariser </li></ul><ul><li>69 texts from CAST corpus were used </li></ul><ul>...
Experiment 1 <ul><li>We evaluated the term-based summariser used in the process </li></ul><ul><li>We found correlation bet...
Experiment 2 <ul><li>Turing-like experiment where we asked humans to pick the better summary in a pair </li></ul><ul><li>E...
Experiment 2 <ul><li>In 41 pairs the summary produced with CAST was preferred </li></ul><ul><li>In 27 pairs the summary pr...
Conclusions <ul><li>Computer-aided summarisation really works for professional summarisers </li></ul><ul><li>and reduces t...
Próximos SlideShares
Carregando em…5
×

What is Computer-Aided Summarisation and does it really work?

Invited talk at Department of linguistics, Tuebingen University, Germany in May 2007

  • Seja o primeiro a comentar

What is Computer-Aided Summarisation and does it really work?

  1. 1. What is computer-aided summarisation and does it really work? Constantin Orasan http://clg.wlv.ac.uk/projects/CAST/
  2. 2. Structure <ul><li>Introduction </li></ul><ul><li>CAST </li></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions </li></ul>
  3. 3. Computer-aided summarisation <ul><li>Combines automatic methods with human input </li></ul><ul><li>Relies on automatic methods to identify the important information </li></ul><ul><li>Humans can decide to include this information and/or additional one </li></ul><ul><li>Humans post-edit the information to produce a coherent summary </li></ul>
  4. 4. <ul><li>Automatic summarisation (AS) </li></ul><ul><li>Produces summaries automatically with the help of computers </li></ul><ul><li>Does not require human input </li></ul><ul><li>The quality is low (especially when compared to human summaries) </li></ul><ul><li>Computer-aided summarisation (CAS) </li></ul><ul><li>Uses automatic methods to produce summaries, but </li></ul><ul><li>Allows the humans to postedit the result </li></ul><ul><li>High quality, but less effort </li></ul>
  5. 5. Why CAS can work? <ul><li>Endres-Niggemeyer (1998) identifies three stages in human summarisation: document exploration , relevance assessment and summary production </li></ul><ul><li>We hypothesise the first two stages can be replaced by automatic methods </li></ul><ul><li>Craven (1996) and Narita (2000) tried to help humans summarisers using automatic means </li></ul>
  6. 6. Computer-aided summarisation tool (CAST) <ul><li>Work funded by Arts and Humanities Research Council </li></ul><ul><li>Work done together with Laura Hasler </li></ul><ul><li>The most important outcome of the project is the tool </li></ul><ul><li>Allows the user to run automatic methods to identify important sentences </li></ul><ul><li>In order to produce an abstract, the user can take sentences and edit them </li></ul>
  7. 7. CAST- the tool (II) <ul><li>At present CAST contains the following methods: </li></ul><ul><ul><li>Keyword method </li></ul></ul><ul><ul><li>Indicating phrases </li></ul></ul><ul><ul><li>Surface clues </li></ul></ul><ul><ul><li>Lexical cohesion </li></ul></ul><ul><li>These methods were chosen because they are highly customisable and domain independent </li></ul><ul><li>The user can select the setting which is the most appropriate for a particular text/genre </li></ul>
  8. 11. Feedback from the user <ul><li>We analysed the work of our human summariser </li></ul><ul><ul><li>Term-based summarisation was used first to produce 30% summaries </li></ul></ul><ul><ul><li>Whenever a useful sentence was found lexical chains were used to identify related sentences </li></ul></ul><ul><ul><li>Avoids to run too many automatic methods because it becomes confusing </li></ul></ul><ul><ul><li>Requested a way to know which sentences have been included in the summary </li></ul></ul>
  9. 12. Evaluation <ul><li>Our assumption about CAS is that it is possible to produce summaries in less time without any loss in quality </li></ul><ul><li>2 experiments were carried out: </li></ul><ul><ul><li>We recorded the time for producing summaries with and without CAST </li></ul></ul><ul><ul><li>Showed pairs of summaries and asked humans to pick the better one </li></ul></ul>
  10. 13. Experiment 1 <ul><li>Used one professional summariser </li></ul><ul><li>69 texts from CAST corpus were used </li></ul><ul><li>Summaries were produced with and without the tool at one year distance </li></ul><ul><li>Without CAST With CAST Reduction % </li></ul><ul><li>Newswire texts 498secs 382secs 23.29% </li></ul><ul><li>New Scientist texts 771secs 623secs 19.19% </li></ul>
  11. 14. Experiment 1 <ul><li>We evaluated the term-based summariser used in the process </li></ul><ul><li>We found correlation between the success of the automatic summariser and the time reduction </li></ul>
  12. 15. Experiment 2 <ul><li>Turing-like experiment where we asked humans to pick the better summary in a pair </li></ul><ul><li>Each pair contained one summary produced with CAST and one without CAST </li></ul><ul><li>17 judges were shown 4 randomly selected pairs </li></ul>
  13. 16. Experiment 2 <ul><li>In 41 pairs the summary produced with CAST was preferred </li></ul><ul><li>In 27 pairs the summary produced without CAST was preferred </li></ul><ul><li>Our assumption was that there is no difference between them </li></ul><ul><li>Chi-square shows that there is no statistically significant difference with 0.05 confidence </li></ul>
  14. 17. Conclusions <ul><li>Computer-aided summarisation really works for professional summarisers </li></ul><ul><li>and reduces the time necessary to produce summaries by about 20% </li></ul><ul><li>It would be interesting to try with non-professional summarisers </li></ul><ul><li>Try on other texts </li></ul><ul><li>Compare to other computer-aided methods </li></ul>

×