summaryrefslogtreecommitdiff
path: root/vendor/bundle/ruby/3.4.0/gems/eventmachine-1.2.7/lib/em/buftok.rb
diff options
context:
space:
mode:
authorBenjamin Sanders <ben@Benjamins-MacBook-Pro.local>2025-10-11 10:34:24 -0400
committerBenjamin Sanders <ben@Benjamins-MacBook-Pro.local>2025-10-11 10:34:24 -0400
commit5571dc766e2143e39762ff0b47c41d1c2e154f4c (patch)
treef4f124d314a7e76c04484515aa0fd1f2e271a601 /vendor/bundle/ruby/3.4.0/gems/eventmachine-1.2.7/lib/em/buftok.rb
parent359f3309c97fc894b63e0a295c76ab298504d635 (diff)
Vendor bundled gems for deployment
Diffstat (limited to 'vendor/bundle/ruby/3.4.0/gems/eventmachine-1.2.7/lib/em/buftok.rb')
-rw-r--r--vendor/bundle/ruby/3.4.0/gems/eventmachine-1.2.7/lib/em/buftok.rb59
1 files changed, 59 insertions, 0 deletions
diff --git a/vendor/bundle/ruby/3.4.0/gems/eventmachine-1.2.7/lib/em/buftok.rb b/vendor/bundle/ruby/3.4.0/gems/eventmachine-1.2.7/lib/em/buftok.rb
new file mode 100644
index 0000000..caf4f77
--- /dev/null
+++ b/vendor/bundle/ruby/3.4.0/gems/eventmachine-1.2.7/lib/em/buftok.rb
@@ -0,0 +1,59 @@
+# BufferedTokenizer takes a delimiter upon instantiation, or acts line-based
+# by default. It allows input to be spoon-fed from some outside source which
+# receives arbitrary length datagrams which may-or-may-not contain the token
+# by which entities are delimited. In this respect it's ideally paired with
+# something like EventMachine (http://rubyeventmachine.com/).
+class BufferedTokenizer
+ # New BufferedTokenizers will operate on lines delimited by a delimiter,
+ # which is by default the global input delimiter $/ ("\n").
+ #
+ # The input buffer is stored as an array. This is by far the most efficient
+ # approach given language constraints (in C a linked list would be a more
+ # appropriate data structure). Segments of input data are stored in a list
+ # which is only joined when a token is reached, substantially reducing the
+ # number of objects required for the operation.
+ def initialize(delimiter = $/)
+ @delimiter = delimiter
+ @input = []
+ @tail = ''
+ @trim = @delimiter.length - 1
+ end
+
+ # Extract takes an arbitrary string of input data and returns an array of
+ # tokenized entities, provided there were any available to extract. This
+ # makes for easy processing of datagrams using a pattern like:
+ #
+ # tokenizer.extract(data).map { |entity| Decode(entity) }.each do ...
+ #
+ # Using -1 makes split to return "" if the token is at the end of
+ # the string, meaning the last element is the start of the next chunk.
+ def extract(data)
+ if @trim > 0
+ tail_end = @tail.slice!(-@trim, @trim) # returns nil if string is too short
+ data = tail_end + data if tail_end
+ end
+
+ @input << @tail
+ entities = data.split(@delimiter, -1)
+ @tail = entities.shift
+
+ unless entities.empty?
+ @input << @tail
+ entities.unshift @input.join
+ @input.clear
+ @tail = entities.pop
+ end
+
+ entities
+ end
+
+ # Flush the contents of the input buffer, i.e. return the input buffer even though
+ # a token has not yet been encountered
+ def flush
+ @input << @tail
+ buffer = @input.join
+ @input.clear
+ @tail = "" # @tail.clear is slightly faster, but not supported on 1.8.7
+ buffer
+ end
+end