The log lines with additional pod metadata such as name IP address and labels. However pod labels may include labels that are use only by Kubernetes controllers and therefore do not provide any value to the user. To avoid wasting resources on processing them we recommend deleting the following labels t reuce inputs logs group_by file stream concat_newline ends_when matche err match.message r if err null false else matche In this case the field merge_strategiesadds a newline character to the message field. In addition the section ends_whenuses a VRL expression to check whether a line ends with a backslash similar to how multiline comments are concatenate in Bash.
Are pruning type pruning
Log collection topologies Now lets talk about log collection topologies that can be use with Vector. Distribute topology. In this case Vector agents are deploye on all nodes of the Kubernetes cluster. Then they collect transform and directly send logs to the storage Nigeria WhatsApp Number List Scheme of operation of a distribute log collection topology Centralize topology. Vector agents also run on all nodes but do not perform complex transformations aggregators do this. The advantage of this type is the preictability of the load. You can deploy deicate nodes for aggregators and if necessary scale them optimizing.
Severity type filter inputs
Vectors resource consumption on cluster nodes Scheme of operation of the centralize log collection topology Stream topology. In it Kubernetes pods get rid of logs as quickly as possible. But writing logs directly to Elasticsearch is slower because it spends time processing them when writing. However if you install an intermeiate buffer for example Kafka then the agents will write Cameroon Phone Number List logs much faster due to the fact that Kafka does not do any processing. And then you can transfer all the logs from Kafka to Elasticsearch using a separate vectorthat is a separate Vector instanceand then conveniently view.
